feat: integrate Runboat E2E testing and performance tests into CI/CD pipeline

Updated .gitlab-ci.yml with complete Phase 3 pipeline stages:

New Stages Added:
- preview: Runboat API call to create ephemeral preview instance
- e2e: Playwright E2E tests against Runboat preview
- performance: Server-side performance benchmarks (latency, queries, tokens)

Pipeline Changes:
- runboat_preview job: Requests preview build, extracts URL, posts MR comment
- e2e_tests job: Runs 19 Playwright scenarios against preview URL
- performance_tests job: Runs 7 performance benchmark tests locally
- All jobs include artifacts (HTML reports, traces) for debugging

Job Dependencies:
- e2e_tests needs runboat_preview (waits for preview URL)
- performance_tests runs in parallel with build stage
- All new jobs only on merge_requests (not main/daily)

New Required CI/CD Variables:
- RUNBOAT_API_URL: Runboat API endpoint (secret)
- RUNBOAT_TOKEN: Bearer token for Runboat (secret)
- GITLAB_BOT_TOKEN: GitLab bot token for MR comments (secret)

Updated PHASE3_ROADMAP.md with:
- Runboat setup instructions
- CI/CD variable requirements and how to obtain
- Complete YAML snippets (already in .gitlab-ci.yml)
- Pipeline flow diagram
- Estimated total pipeline time: ~35 minutes

Non-blocking failures:
- runboat_preview: allow_failure=true (Runboat might be unavailable)
- e2e_tests: allow_failure=true (E2E informational, doesn't block merge)
- performance_tests: allow_failure=false (must pass)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

.gitlab-ci.yml

@@ -2,6 +2,8 @@ stages:
   - lint
   - test
   - build
+  - preview
+  - e2e
   - notify
 
 variables:
@@ -88,6 +90,120 @@ unit_tests:
     expire_in: 30 days
   coverage: '/^TOTAL.*\s+(\d+%)$/'
 
+# ================================================================
+# RUNBOAT PREVIEW DEPLOYMENT
+# ================================================================
+
+runboat_preview:
+  stage: preview
+  image: curlimages/curl:latest
+  script:
+    # Request preview build from Runboat
+    - |
+      RESP=$(curl -fsSL -X POST "${RUNBOAT_API_URL}/builds" \
+        -H "Authorization: Bearer ${RUNBOAT_TOKEN}" \
+        -H "Content-Type: application/json" \
+        -d "{
+          \"repo\": \"${CI_PROJECT_PATH}\",
+          \"sha\": \"${CI_COMMIT_SHA}\",
+          \"target_branch\": \"${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-main}\"
+        }")
+    # Extract preview URL
+    - BUILD_URL=$(echo "$RESP" | jq -r '.url')
+    - echo "BUILD_URL=$BUILD_URL" >> build.env
+    # Post comment to MR with preview link
+    - |
+      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
+        curl -X POST "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes" \
+          -H "PRIVATE-TOKEN: ${GITLAB_BOT_TOKEN}" \
+          -d "body=🚀 [Preview](${BUILD_URL}/odoo) ready for E2E testing"
+      fi
+  artifacts:
+    reports:
+      dotenv: build.env
+  only:
+    - merge_requests
+  allow_failure: true  # Runboat might be unavailable, don't block MR
+
+# ================================================================
+# E2E TESTS (Runboat Preview)
+# ================================================================
+
+e2e_tests:
+  stage: e2e
+  image: mcr.microsoft.com/playwright/python:latest
+  needs:
+    - runboat_preview
+  before_script:
+    - pip install --no-cache-dir -r e2e/requirements.txt
+  script:
+    # Run E2E tests against Runboat preview
+    - |
+      pytest e2e/ \
+        --base-url="${BUILD_URL}" \
+        -v \
+        --tb=short \
+        --tracing=retain-on-failure \
+        --html=report-e2e.html \
+        --self-contained-html
+  artifacts:
+    when: always
+    paths:
+      - e2e/traces/
+      - report-e2e.html
+    expire_in: 1 week
+  only:
+    - merge_requests
+  allow_failure: true  # E2E failures don't block merge (informational)
+
+# ================================================================
+# PERFORMANCE TESTS (Local)
+# ================================================================
+
+performance_tests:
+  stage: test
+  image: $ODOO_IMAGE
+  services:
+    - name: postgres:15
+      alias: postgres
+      command:
+        - postgres
+        - -c
+        - fsync=off
+        - -c
+        - shared_buffers=512MB
+  before_script:
+    # Create primed template database
+    - createdb -h $POSTGRES_HOST -U $POSTGRES_USER $TEMPLATE_DATABASE
+    # Install Odoo + modules
+    - |
+      odoo -d $TEMPLATE_DATABASE \
+        -i base,website,website_blog,mail,itsulu_blog_publisher \
+        --addons-path=/usr/lib/python3/dist-packages/odoo/addons,/builds/$CI_PROJECT_PATH/addons \
+        --without-demo=all --stop-after-init --db_host=$POSTGRES_HOST --db_user=$POSTGRES_USER
+    # Install test dependencies
+    - pip install --no-cache-dir pytest pytest-odoo pytest-cov pytest-html
+  script:
+    # Clone template DB for test isolation
+    - createdb -h $POSTGRES_HOST -U $POSTGRES_USER -T $TEMPLATE_DATABASE $POSTGRES_DB
+    # Run performance tests
+    - |
+      python3 -m pytest \
+        addons/itsulu_blog_publisher/tests/test_performance.py \
+        -v \
+        -m performance \
+        --odoo-database=$POSTGRES_DB \
+        --html=report-performance.html \
+        --self-contained-html
+  artifacts:
+    when: always
+    paths:
+      - report-performance.html
+    expire_in: 30 days
+  only:
+    - merge_requests
+  allow_failure: false  # Performance tests must pass
+
 # ================================================================
 # BUILD DOCKER IMAGE
 # ================================================================

PHASE3_ROADMAP.md

@@ -194,59 +194,95 @@ Runboat (by Acsone) provides:
 - **Fresh template DB** with addon pre-installed
 - **5-minute auto-cleanup** after test run
 
-### CI/CD Integration
+### CI/CD Variables Required
 
+Add these to **GitLab Project Settings → CI/CD Variables**:
+
+| Variable | Type | Purpose | Example |
+|----------|------|---------|---------|
+| `RUNBOAT_API_URL` | Secret | Runboat API endpoint | `https://api.runboat.dev` |
+| `RUNBOAT_TOKEN` | Secret | Bearer token for Runboat API | `rbk_xxx...` |
+| `GITLAB_BOT_TOKEN` | Secret | Personal/bot token for MR comments | `glpat_xxx...` |
+
+**How to obtain**:
+1. **RUNBOAT_API_URL & RUNBOAT_TOKEN**: Request from Acsone/infrastructure team
+2. **GITLAB_BOT_TOKEN**: Create via **GitLab → Settings → Access Tokens**
+   - Scopes: `api`, `read_api`, `read_repository`
+   - Save as CI/CD variable (marked as Protected, Masked)
+
+### CI/CD Integration (Already Added)
+
+`.gitlab-ci.yml` now includes:
+
+**Stage: preview**
 ```yaml
-# .gitlab-ci.yml
-
-stages: [lint, test, build, preview, e2e]
-
-# ... existing lint + test stages ...
-
 runboat_preview:
   stage: preview
   image: curlimages/curl:latest
   script:
-    - |
+    # Request preview build from Runboat
-      RUNBOAT_URL="${RUNBOAT_API_URL}/builds"
+    - RESP=$(curl -fsSL -X POST "${RUNBOAT_API_URL}/builds" \
-      RESP=$(curl -fsSL -X POST "$RUNBOAT_URL" \
+        -H "Authorization: Bearer ${RUNBOAT_TOKEN}" \
-        -H "Authorization: Bearer $RUNBOAT_TOKEN" \
+        -d "{\"repo\":\"${CI_PROJECT_PATH}\",\"sha\":\"${CI_COMMIT_SHA}\"}")
-        -H "Content-Type: application/json" \
+    - BUILD_URL=$(echo "$RESP" | jq -r '.url')
-        -d "{
+    - echo "BUILD_URL=$BUILD_URL" >> build.env
-          \"repo\": \"$CI_PROJECT_PATH\",
+    # Post comment to MR
-          \"sha\": \"$CI_COMMIT_SHA\",
+    - curl -X POST "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes" \
-          \"target_branch\": \"$CI_MERGE_REQUEST_TARGET_BRANCH_NAME\"
+        -H "PRIVATE-TOKEN: ${GITLAB_BOT_TOKEN}" \
-        }")
+        -d "body=🚀 [Preview](${BUILD_URL}/odoo) ready"
-      BUILD_URL=$(echo "$RESP" | jq -r '.url')
-      echo "BUILD_URL=$BUILD_URL" >> build.env
-      
-      # Post comment to MR
-      curl -X POST "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes" \
-        -H "PRIVATE-TOKEN: $GITLAB_BOT_TOKEN" \
-        --data "body=🚀 [Preview](${BUILD_URL}/odoo) ready for testing"
   artifacts:
     reports:
       dotenv: build.env
-  rules:
+```
-    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
 
+**Stage: e2e**
+```yaml
 e2e_tests:
   stage: e2e
   image: mcr.microsoft.com/playwright/python:latest
-  needs:
+  needs: [runboat_preview]
-    - runboat_preview
   script:
     - pip install -r e2e/requirements.txt
     - pytest e2e/ --base-url=$BUILD_URL -v --tracing=retain-on-failure
   artifacts:
-    when: on_failure
+    when: always
     paths:
       - e2e/traces/
     expire_in: 1 week
-  rules:
-    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
 ```
 
+**Stage: test (performance)**
+```yaml
+performance_tests:
+  stage: test
+  image: $ODOO_IMAGE
+  script:
+    - pytest addons/itsulu_blog_publisher/tests/test_performance.py \
+        -m performance --odoo-database=$POSTGRES_DB
+```
+
+### Pipeline Flow
+
+```
+Merge Request
+    ↓
+[lint] black, pylint-odoo (2 min)
+    ↓
+[test] unit + BDD + performance (10 min)
+    ↓
+[build] Docker image → registry (3 min)
+    ↓
+[preview] Runboat deploy (5 min)
+    ↓
+[e2e] Playwright against preview (15 min)
+    ↓
+Results → MR comment with preview URL
+```
+
+**Total pipeline time**: ~35 minutes
+- Unit/BDD/Performance tests run in parallel with Docker build
+- E2E tests run after preview is ready
+
 ## Success Criteria
 
 ✅ **Phase 3 Complete when:**