feat: integrate Runboat E2E testing and performance tests into CI/CD pipeline

Updated .gitlab-ci.yml with complete Phase 3 pipeline stages: New Stages Added: - preview: Runboat API call to create ephemeral preview instance - e2e: Playwright E2E tests against Runboat preview - performance: Server-side performance benchmarks (latency, queries, tokens) Pipeline Changes: - runboat_preview job: Requests preview build, extracts URL, posts MR comment - e2e_tests job: Runs 19 Playwright scenarios against preview URL - performance_tests job: Runs 7 performance benchmark tests locally - All jobs include artifacts (HTML reports, traces) for debugging Job Dependencies: - e2e_tests needs runboat_preview (waits for preview URL) - performance_tests runs in parallel with build stage - All new jobs only on merge_requests (not main/daily) New Required CI/CD Variables: - RUNBOAT_API_URL: Runboat API endpoint (secret) - RUNBOAT_TOKEN: Bearer token for Runboat (secret) - GITLAB_BOT_TOKEN: GitLab bot token for MR comments (secret) Updated PHASE3_ROADMAP.md with: - Runboat setup instructions - CI/CD variable requirements and how to obtain - Complete YAML snippets (already in .gitlab-ci.yml) - Pipeline flow diagram - Estimated total pipeline time: ~35 minutes Non-blocking failures: - runboat_preview: allow_failure=true (Runboat might be unavailable) - e2e_tests: allow_failure=true (E2E informational, doesn't block merge) - performance_tests: allow_failure=false (must pass) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-05-30 23:41:23 +00:00 · 2026-05-30 00:54:59 -04:00 · 2026-05-30 00:54:59 -04:00 · d122b773d4
commit d122b773d4
parent 11918da2ea
2 changed files with 183 additions and 31 deletions
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@ -2,6 +2,8 @@ stages:
  - lint
  - test
  - build
+  - preview
+  - e2e
  - notify

 variables:
@ -88,6 +90,120 @@ unit_tests:
    expire_in: 30 days
  coverage: '/^TOTAL.*\s+(\d+%)$/'

+# ================================================================
+# RUNBOAT PREVIEW DEPLOYMENT
+# ================================================================
+
+runboat_preview:
+  stage: preview
+  image: curlimages/curl:latest
+  script:
+    # Request preview build from Runboat
+    - |
+      RESP=$(curl -fsSL -X POST "${RUNBOAT_API_URL}/builds" \
+        -H "Authorization: Bearer ${RUNBOAT_TOKEN}" \
+        -H "Content-Type: application/json" \
+        -d "{
+          \"repo\": \"${CI_PROJECT_PATH}\",
+          \"sha\": \"${CI_COMMIT_SHA}\",
+          \"target_branch\": \"${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-main}\"
+        }")
+    # Extract preview URL
+    - BUILD_URL=$(echo "$RESP" | jq -r '.url')
+    - echo "BUILD_URL=$BUILD_URL" >> build.env
+    # Post comment to MR with preview link
+    - |
+      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
+        curl -X POST "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes" \
+          -H "PRIVATE-TOKEN: ${GITLAB_BOT_TOKEN}" \
+          -d "body=🚀 [Preview](${BUILD_URL}/odoo) ready for E2E testing"
+      fi
+  artifacts:
+    reports:
+      dotenv: build.env
+  only:
+    - merge_requests
+  allow_failure: true  # Runboat might be unavailable, don't block MR
+
+# ================================================================
+# E2E TESTS (Runboat Preview)
+# ================================================================
+
+e2e_tests:
+  stage: e2e
+  image: mcr.microsoft.com/playwright/python:latest
+  needs:
+    - runboat_preview
+  before_script:
+    - pip install --no-cache-dir -r e2e/requirements.txt
+  script:
+    # Run E2E tests against Runboat preview
+    - |
+      pytest e2e/ \
+        --base-url="${BUILD_URL}" \
+        -v \
+        --tb=short \
+        --tracing=retain-on-failure \
+        --html=report-e2e.html \
+        --self-contained-html
+  artifacts:
+    when: always
+    paths:
+      - e2e/traces/
+      - report-e2e.html
+    expire_in: 1 week
+  only:
+    - merge_requests
+  allow_failure: true  # E2E failures don't block merge (informational)
+
+# ================================================================
+# PERFORMANCE TESTS (Local)
+# ================================================================
+
+performance_tests:
+  stage: test
+  image: $ODOO_IMAGE
+  services:
+    - name: postgres:15
+      alias: postgres
+      command:
+        - postgres
+        - -c
+        - fsync=off
+        - -c
+        - shared_buffers=512MB
+  before_script:
+    # Create primed template database
+    - createdb -h $POSTGRES_HOST -U $POSTGRES_USER $TEMPLATE_DATABASE
+    # Install Odoo + modules
+    - |
+      odoo -d $TEMPLATE_DATABASE \
+        -i base,website,website_blog,mail,itsulu_blog_publisher \
+        --addons-path=/usr/lib/python3/dist-packages/odoo/addons,/builds/$CI_PROJECT_PATH/addons \
+        --without-demo=all --stop-after-init --db_host=$POSTGRES_HOST --db_user=$POSTGRES_USER
+    # Install test dependencies
+    - pip install --no-cache-dir pytest pytest-odoo pytest-cov pytest-html
+  script:
+    # Clone template DB for test isolation
+    - createdb -h $POSTGRES_HOST -U $POSTGRES_USER -T $TEMPLATE_DATABASE $POSTGRES_DB
+    # Run performance tests
+    - |
+      python3 -m pytest \
+        addons/itsulu_blog_publisher/tests/test_performance.py \
+        -v \
+        -m performance \
+        --odoo-database=$POSTGRES_DB \
+        --html=report-performance.html \
+        --self-contained-html
+  artifacts:
+    when: always
+    paths:
+      - report-performance.html
+    expire_in: 30 days
+  only:
+    - merge_requests
+  allow_failure: false  # Performance tests must pass
+
 # ================================================================
 # BUILD DOCKER IMAGE
 # ================================================================
--- a/PHASE3_ROADMAP.md
+++ b/PHASE3_ROADMAP.md
@ -194,59 +194,95 @@ Runboat (by Acsone) provides:
 - **Fresh template DB** with addon pre-installed
 - **5-minute auto-cleanup** after test run

-### CI/CD Integration
+### CI/CD Variables Required

+Add these to **GitLab Project Settings → CI/CD Variables**:
+
+| Variable | Type | Purpose | Example |
+|----------|------|---------|---------|
+| `RUNBOAT_API_URL` | Secret | Runboat API endpoint | `https://api.runboat.dev` |
+| `RUNBOAT_TOKEN` | Secret | Bearer token for Runboat API | `rbk_xxx...` |
+| `GITLAB_BOT_TOKEN` | Secret | Personal/bot token for MR comments | `glpat_xxx...` |
+
+**How to obtain**:
+1. **RUNBOAT_API_URL & RUNBOAT_TOKEN**: Request from Acsone/infrastructure team
+2. **GITLAB_BOT_TOKEN**: Create via **GitLab → Settings → Access Tokens**
+   - Scopes: `api`, `read_api`, `read_repository`
+   - Save as CI/CD variable (marked as Protected, Masked)
+
+### CI/CD Integration (Already Added)
+
+`.gitlab-ci.yml` now includes:
+
+**Stage: preview**
 ```yaml
-# .gitlab-ci.yml
-
-stages: [lint, test, build, preview, e2e]
-
-# ... existing lint + test stages ...
-
 runboat_preview:
  stage: preview
  image: curlimages/curl:latest
  script:
-    - |
-      RUNBOAT_URL="${RUNBOAT_API_URL}/builds"
-      RESP=$(curl -fsSL -X POST "$RUNBOAT_URL" \
-        -H "Authorization: Bearer $RUNBOAT_TOKEN" \
-        -H "Content-Type: application/json" \
-        -d "{
-          \"repo\": \"$CI_PROJECT_PATH\",
-          \"sha\": \"$CI_COMMIT_SHA\",
-          \"target_branch\": \"$CI_MERGE_REQUEST_TARGET_BRANCH_NAME\"
-        }")
-      BUILD_URL=$(echo "$RESP" | jq -r '.url')
-      echo "BUILD_URL=$BUILD_URL" >> build.env
-      
+    # Request preview build from Runboat
+    - RESP=$(curl -fsSL -X POST "${RUNBOAT_API_URL}/builds" \
+        -H "Authorization: Bearer ${RUNBOAT_TOKEN}" \
+        -d "{\"repo\":\"${CI_PROJECT_PATH}\",\"sha\":\"${CI_COMMIT_SHA}\"}")
+    - BUILD_URL=$(echo "$RESP" | jq -r '.url')
+    - echo "BUILD_URL=$BUILD_URL" >> build.env
    # Post comment to MR
-      curl -X POST "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes" \
-        -H "PRIVATE-TOKEN: $GITLAB_BOT_TOKEN" \
-        --data "body=🚀 [Preview](${BUILD_URL}/odoo) ready for testing"
+    - curl -X POST "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes" \
+        -H "PRIVATE-TOKEN: ${GITLAB_BOT_TOKEN}" \
+        -d "body=🚀 [Preview](${BUILD_URL}/odoo) ready"
  artifacts:
    reports:
      dotenv: build.env
-  rules:
-    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+```

+**Stage: e2e**
+```yaml
 e2e_tests:
  stage: e2e
  image: mcr.microsoft.com/playwright/python:latest
-  needs:
-    - runboat_preview
+  needs: [runboat_preview]
  script:
    - pip install -r e2e/requirements.txt
    - pytest e2e/ --base-url=$BUILD_URL -v --tracing=retain-on-failure
  artifacts:
-    when: on_failure
+    when: always
    paths:
      - e2e/traces/
    expire_in: 1 week
-  rules:
-    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
 ```

+**Stage: test (performance)**
+```yaml
+performance_tests:
+  stage: test
+  image: $ODOO_IMAGE
+  script:
+    - pytest addons/itsulu_blog_publisher/tests/test_performance.py \
+        -m performance --odoo-database=$POSTGRES_DB
+```
+
+### Pipeline Flow
+
+```
+Merge Request
+    ↓
+[lint] black, pylint-odoo (2 min)
+    ↓
+[test] unit + BDD + performance (10 min)
+    ↓
+[build] Docker image → registry (3 min)
+    ↓
+[preview] Runboat deploy (5 min)
+    ↓
+[e2e] Playwright against preview (15 min)
+    ↓
+Results → MR comment with preview URL
+```
+
+**Total pipeline time**: ~35 minutes
+- Unit/BDD/Performance tests run in parallel with Docker build
+- E2E tests run after preview is ready
+
 ## Success Criteria

 ✅ **Phase 3 Complete when:**