From 11918da2ea23dbfed5d8049bb39ff18be9733268 Mon Sep 17 00:00:00 2001
From: Nicholas Riegel <nicholasr@itsulu.com>
Date: Sat, 30 May 2026 00:52:37 -0400
Subject: [PATCH] docs: add E2E testing and performance SLO documentation to
 CLAUDE.md

Added comprehensive sections documenting Phase 3 testing:

Section 7.4 - Runboat Integration:
- Explanation of Runboat (ephemeral preview instances)
- Cold-start polling pattern with 180s timeout
- CI/CD integration example with buildenv artifact
- E2E test invocation against Runboat URL

Section 13 - Performance SLO Targets:
- Test infrastructure SLOs (pipeline times, test coverage, flakiness)
- Generation performance targets:
  * Latency P50: <30s, P99: <60s
  * Token efficiency: 800-1200 per post
  * Query count: <50 per generation
  * Concurrent posts: 5+
  * Email latency: <5s
  * Template DB prime: <60s
- Measurement tools and patterns:
  * time.monotonic() for latency profiling
  * assertQueryCount() for N+1 detection
  * Token usage logging and assertions

These targets are verified by test_performance.py and E2E tests.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
---
 CLAUDE.md | 102 ++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 91 insertions(+), 11 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index 8005909..9d1a562 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -406,18 +406,63 @@ def page(browser, browser_context_args, auth_state):
 ### 7.3 Example scenario
 
 ```python
-# e2e/test_sale_order.py
+# e2e/test_generation.py
 from playwright.sync_api import expect
 
-def test_gold_customer_discount_shows_in_ui(page):
-    page.goto('/odoo/sales/new')
-    page.get_by_label('Customer').fill('Acme Corp')
-    page.get_by_role('option', name='Acme Corp').click()
-    page.get_by_role('button', name='Add a product').click()
-    page.get_by_label('Product').last.fill('Widget')
-    page.get_by_role('option', name='Widget').click()
+def test_user_generates_blog_post_on_demand(page):
+    """Navigate to Blog Publisher, fill form, generate post, verify published."""
+    page.goto('/odoo/blog/generate-now')
+    page.wait_for_load_state('networkidle')
+    
+    page.get_by_label('Topic').fill('Kubernetes Cost Optimization')
+    page.get_by_label('LLM Provider').select_option('anthropic')
+    page.get_by_label('Auto-publish').check()
+    
+    page.get_by_role('button', name='Generate').click()
+    page.wait_for_url('**/blog/**', timeout=60_000)
+    
+    expect(page.locator('h1')).to_contain_text('Kubernetes')
+```
 
-    expect(page.locator('[data-test-id="line-discount"]').first).to_have_text('10.00')
+### 7.4 Runboat Integration
+
+**Runboat** provides ephemeral preview instances of Odoo per CI commit:
+- **Auto-deployment**: Fresh Odoo instance with addon pre-installed
+- **Live URL**: For E2E tests (no local bootstrapping)
+- **Auto-cleanup**: Instance removed 5 minutes after test completion
+- **Template DB**: Primed once, cloned for each test
+
+**Cold-start Handling**:
+```python
+def wait_for_odoo(url, timeout=180):
+    """Poll until Odoo responds (instance startup takes 30-60s)."""
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        try:
+            if requests.get(f"{url}/web/login", timeout=5).status_code == 200:
+                return
+        except Exception:
+            pass
+        time.sleep(2)
+    raise TimeoutError(f"Odoo not ready after {timeout}s")
+```
+
+**CI/CD Integration**:
+```yaml
+# .gitlab-ci.yml
+runboat_preview:
+  stage: preview
+  script: |
+    RESP=$(curl -fsSL -X POST $RUNBOAT_URL/builds \
+      -H "Authorization: Bearer $RUNBOAT_TOKEN" \
+      -d "{\"repo\":\"$CI_PROJECT_PATH\",\"sha\":\"$CI_COMMIT_SHA\"}")
+    BUILD_URL=$(echo "$RESP" | jq -r '.url')
+    echo "BUILD_URL=$BUILD_URL" >> build.env
+
+e2e_tests:
+  stage: e2e
+  needs: [runboat_preview]
+  script: pytest e2e/ --base-url=$BUILD_URL -v
 ```
 
 ---
@@ -813,7 +858,9 @@ post = self.env['blog.post'].sudo().create({
 
 ---
 
-## 13. SLO Targets
+## 13. Performance SLO Targets
+
+### Test Infrastructure
 
 | Metric | Target | Why |
 |---|---|---|
@@ -823,7 +870,40 @@ post = self.env['blog.post'].sudo().create({
 | Flaky test rate | < 2%/week | Maintains trust in suite |
 | Coverage on new code | ≥ 80% | Enforces test-first habit |
 | Runboat cold-start P95 | < 120 s | E2E does not time out |
-| Production deploy MTTR | < 15 min | Git revert + ArgoCD sync |
+
+### Generation Performance (Phase 3)
+
+| Metric | Target | Measurement |
+|---|---|---|
+| Generation latency P50 | < 30 seconds | RED → POST created |
+| Generation latency P99 | < 60 seconds | 99th percentile |
+| Tokens per post | 800–1200 | Cost baseline |
+| Queries per generation | < 50 | N+1 detection |
+| Concurrent posts | 5+ | Peak throughput |
+| Email send latency | < 5 seconds | Notification speed |
+| Template DB prime | < 60 seconds | CI/CD overhead |
+
+**Measurement Tools**:
+```python
+# Latency profiling
+import time
+start = time.monotonic()
+post = schedule.run_generation()
+elapsed = time.monotonic() - start
+assert elapsed < 30  # P50 target
+
+# Query count assertion
+with self.assertQueryCount(50):
+    schedule.run_generation()  # Must use < 50 queries
+
+# Token usage logging
+log.tokens_used  # Recorded in generation log
+assert 800 <= log.tokens_used <= 1200
+```
+
+---
+
+## 14. Production Deploy
 
 ---