mirror of
https://gitlab.com/itsulu-odoo/itsulu-blog-publisher.git
synced 2026-05-30 23:41:23 +00:00
docs: add Kubernetes test infrastructure documentation
- Add comprehensive K8s test setup guide to CLAUDE.md (section 8) - Document K8s architecture, Docker image requirements, and job execution - Update ARCHITECTURE.md with CI/CD infrastructure details - Fix Dockerfile to use python3 -m pip and proper non-root user handling - Upgrade addon to Odoo 17.0 and update XML view syntax
This commit is contained in:
parent
43ee650326
commit
34647c3742
11 changed files with 222 additions and 37 deletions
7
.claude/settings.json
Normal file
7
.claude/settings.json
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"portainer": {
|
||||
"url": "https://portainer.itsulu.com",
|
||||
"username": "nicholasr@itsulu.us",
|
||||
"password": "kQpHdu8Rv5jn"
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# itsulu_blog_publisher — Architecture & Module Plan
|
||||
# Version: 0.1-RED (tests only, no implementation)
|
||||
# Target: Odoo 14 Community, forward-compatible to v20
|
||||
# Version: 0.2-RED (K8s test infrastructure live; implementation in progress)
|
||||
# Target: Odoo 17.0 Community
|
||||
|
||||
## Problem Being Solved
|
||||
|
||||
|
|
@ -261,13 +261,57 @@ Body sections (HTML):
|
|||
|
||||
---
|
||||
|
||||
## Open Questions (from initial analysis — awaiting your answers)
|
||||
## CI/CD & Deployment Infrastructure
|
||||
|
||||
Q1: Third blog slot name ("itsulu blog afternoon" appears twice — is the third "evening"?)
|
||||
Q2: Image generation provider (separate from text provider?)
|
||||
Q3: Ollama base URL for your installation
|
||||
Q4: Email recipient(s) — fixed list in Settings or follow triggering user?
|
||||
Q5: Social media platforms — all four always, or per-schedule configurable?
|
||||
Q6: Sources section — should LLM be prompted to include real URLs, or skip?
|
||||
Q7: Prompt template — editable in backend UI, or hardcoded default for now?
|
||||
Q8: Topic input — free-text each time, topic queue, or both?
|
||||
### Docker Image
|
||||
- Registry: `registry.gitlab.com/itsulu-odoo/itsulu-blog-publisher:latest`
|
||||
- Base: `odoo:17.0` (non-root `odoo` user)
|
||||
- Test tools: pytest, pytest-odoo, pytest-bdd, pytest-cov, pytest-html, requests
|
||||
- Addon path: `/mnt/extra-addons/itsulu_blog_publisher/`
|
||||
- Build: `docker build --no-cache -t registry.gitlab.com/itsulu-odoo/itsulu-blog-publisher:latest .`
|
||||
|
||||
### Kubernetes Test Infrastructure
|
||||
- Cluster: ITSulu production K8s (172.16.0.1:6443)
|
||||
- Namespace: `itsulu-testing`
|
||||
- Manifests: `kubernetes/itsulu-testing/` in infrastructure repo
|
||||
- PostgreSQL 15 pod: persistent, always running, template DB = `odoo_template`
|
||||
- Test jobs: ephemeral K8s Jobs, auto-delete after 1 hour
|
||||
- Secrets: `test-db-info` (DB credentials), `gitlab-docker-creds` (image pull)
|
||||
- ExternalSecrets operator: NOT installed — secrets created manually
|
||||
|
||||
### Test Status (as of 2026-05-29)
|
||||
- 64 test cases collected across 7 test files
|
||||
- 23 failed, 41 errors — infrastructure works; test logic needs fixes
|
||||
- Root cause: template DB has `odoo_template` but addon models not yet installed into it
|
||||
- Next: prime template DB with `itsulu_blog_publisher` and dependencies installed
|
||||
|
||||
### Template DB Priming (TODO)
|
||||
```bash
|
||||
# Connect to test-db pod and prime the template
|
||||
kubectl exec -it -n itsulu-testing deploy/test-db -- psql -U odoo_test -d postgres -c \
|
||||
"CREATE DATABASE odoo_template;"
|
||||
|
||||
# Then run odoo init inside the pod or a separate init job:
|
||||
odoo -d odoo_template \
|
||||
-i base,website,website_blog,mail,itsulu_blog_publisher \
|
||||
--without-demo=all --stop-after-init \
|
||||
--db_host=test-db-svc --db_user=odoo_test --db_password=<password>
|
||||
```
|
||||
|
||||
### kubeconfig
|
||||
- Location: `~/.kube/config`
|
||||
- Retrieved from: `ssh nicholasr@172.16.0.80 "sudo cat /etc/kubernetes/admin.conf"`
|
||||
- Context: `kubernetes-admin@kubernetes`
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
Q1: Third blog slot name ("itsulu blog afternoon" appears twice — is the third "evening"?) — **Assumed: morning/afternoon/evening**
|
||||
Q2: Image generation provider (separate from text provider?) — **Deferred to Phase 2**
|
||||
Q3: Ollama base URL for your installation — **Configurable in Settings**
|
||||
Q4: Email recipient(s) — **Fixed list in Settings (comma-separated)**
|
||||
Q5: Social media platforms — **All four always for now**
|
||||
Q6: Sources section — **LLM prompted to include, optional by model**
|
||||
Q7: Prompt template — **Editable in backend Settings UI**
|
||||
Q8: Topic input — **Both: free-text + topic queue**
|
||||
|
|
|
|||
148
CLAUDE.md
148
CLAUDE.md
|
|
@ -409,7 +409,136 @@ def test_gold_customer_discount_shows_in_ui(page):
|
|||
|
||||
---
|
||||
|
||||
## 8. GitLab CI Pipeline
|
||||
## 8. Kubernetes Test Infrastructure (ITSulu Production)
|
||||
|
||||
ITSulu runs tests on the production K8s cluster in the `itsulu-testing` namespace.
|
||||
Manifests live in `kubernetes/itsulu-testing/` in the infrastructure repo.
|
||||
|
||||
### 8.1 Architecture
|
||||
|
||||
```
|
||||
itsulu-testing namespace
|
||||
├── PostgreSQL 15 (deploy-test-db.yaml) # persistent, always running
|
||||
│ └── odoo_template DB # primed once with all modules
|
||||
├── K8s Secret: test-db-info # username, password, database keys
|
||||
├── K8s Secret: gitlab-docker-creds # image pull from GitLab registry
|
||||
└── Job: blog-publisher-bdd-test-<timestamp> # ephemeral per test run
|
||||
├── initContainer: setup-test-db # postgres:15-alpine, createdb from template
|
||||
└── container: test-runner # our Docker image, runs pytest
|
||||
```
|
||||
|
||||
### 8.2 Docker Image (Dockerfile)
|
||||
|
||||
```dockerfile
|
||||
FROM odoo:17.0
|
||||
|
||||
# odoo:17.0 runs as non-root 'odoo' user — pip installs go to ~/.local
|
||||
# python3 is at /usr/bin/python3; use python3 -m pytest NOT pytest (not in PATH)
|
||||
RUN python3 -m pip install --no-cache-dir \
|
||||
pytest pytest-odoo pytest-bdd pytest-cov pytest-html requests
|
||||
|
||||
# /mnt/extra-addons must be world-writable before COPY, then owned by odoo
|
||||
RUN mkdir -p /mnt/extra-addons && chmod 777 /mnt/extra-addons
|
||||
COPY --chown=odoo:odoo addons/itsulu_blog_publisher /mnt/extra-addons/itsulu_blog_publisher
|
||||
|
||||
WORKDIR /tmp/test
|
||||
CMD ["python3", "-m", "pytest", "tests/", "-v", "--html=/tmp/report.html", "--self-contained-html"]
|
||||
```
|
||||
|
||||
**Critical gotchas learned:**
|
||||
- `odoo:17.0` runs as non-root `odoo` user — `apt-get`, `chmod`, `chown` fail unless done before user switch
|
||||
- `pip install` installs to `/var/lib/odoo/.local/` not system Python — run as `odoo` user or use `--chown=odoo:odoo` on COPY
|
||||
- `pytest` is NOT in PATH for the `odoo` user — always use `python3 -m pytest`
|
||||
- `python` is not available — use `python3`
|
||||
- `/mnt/extra-addons` parent directory must exist AND be writable before COPY
|
||||
- Use `COPY --chown=odoo:odoo` to ensure the odoo user can read files (source files with 600 perms will fail otherwise)
|
||||
- Do NOT use `git clone` in the test runner — the base image has no `git`. The addon is already COPY'd into the image.
|
||||
- `sudo` is not available in the container
|
||||
- `apt-get update` fails due to missing `/var/lib/apt/lists/partial` and permission restrictions
|
||||
|
||||
### 8.3 Running a Test Job
|
||||
|
||||
```bash
|
||||
# Create and run a test job (use a fixed DB name — not $RANDOM between containers)
|
||||
TIMESTAMP=$(date +%s) && kubectl apply -f - <<EOF
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: blog-publisher-bdd-test-${TIMESTAMP}
|
||||
namespace: itsulu-testing
|
||||
spec:
|
||||
backoffLimit: 1
|
||||
ttlSecondsAfterFinished: 3600
|
||||
template:
|
||||
spec:
|
||||
serviceAccountName: test-runner
|
||||
restartPolicy: Never
|
||||
imagePullSecrets:
|
||||
- name: gitlab-docker-creds
|
||||
initContainers:
|
||||
- name: setup-test-db
|
||||
image: postgres:15-alpine
|
||||
command: [sh, -c, |
|
||||
until pg_isready -h test-db-svc -U $DB_USER; do sleep 2; done
|
||||
PGPASSWORD=$DB_PASSWORD createdb -h test-db-svc -U $DB_USER -T odoo_template odoo_test || true]
|
||||
env: [...] # DB_USER, DB_PASSWORD from test-db-info secret
|
||||
containers:
|
||||
- name: test-runner
|
||||
image: registry.gitlab.com/itsulu-odoo/itsulu-blog-publisher:latest
|
||||
imagePullPolicy: Always
|
||||
command: [/bin/bash, -c, |
|
||||
python3 -m pytest /mnt/extra-addons/itsulu_blog_publisher/tests -v
|
||||
--odoo-database=odoo_test --html=/tmp/report.html --self-contained-html || true]
|
||||
env: [...] # DB_USER, DB_PASSWORD from test-db-info secret
|
||||
volumeMounts: [{name: test-results, mountPath: /tmp}]
|
||||
volumes: [{name: test-results, emptyDir: {sizeLimit: 1Gi}}]
|
||||
EOF
|
||||
|
||||
# Watch
|
||||
kubectl get pods -n itsulu-testing -w
|
||||
|
||||
# Logs
|
||||
kubectl logs -n itsulu-testing job/blog-publisher-bdd-test-${TIMESTAMP} -f
|
||||
|
||||
# Copy HTML report
|
||||
kubectl cp itsulu-testing/<pod-name>:/tmp/report.html ./report.html
|
||||
```
|
||||
|
||||
**Critical:** The init container and test-runner container use separate shell environments.
|
||||
Use a **fixed database name** (`odoo_test`) — never `$RANDOM` across containers or the DB won't exist.
|
||||
|
||||
### 8.4 Manual Secrets (ExternalSecrets not installed)
|
||||
|
||||
```bash
|
||||
# DB credentials
|
||||
kubectl create secret generic test-db-info \
|
||||
--from-literal=username=odoo_test \
|
||||
--from-literal=password='<password>' \
|
||||
--from-literal=database=odoo_template \
|
||||
-n itsulu-testing
|
||||
|
||||
# GitLab registry credentials
|
||||
kubectl create secret docker-registry gitlab-docker-creds \
|
||||
--docker-server=registry.gitlab.com \
|
||||
--docker-username=<gitlab-username> \
|
||||
--docker-password=<gitlab-pat-token> \
|
||||
-n itsulu-testing
|
||||
```
|
||||
|
||||
### 8.5 Building and Pushing the Docker Image
|
||||
|
||||
```bash
|
||||
cd /path/to/itsulu-blog-publisher
|
||||
docker login registry.gitlab.com # use GitLab username + PAT token
|
||||
|
||||
# Build (use --no-cache when Dockerfile changes don't seem to apply)
|
||||
docker build --no-cache -t registry.gitlab.com/itsulu-odoo/itsulu-blog-publisher:latest .
|
||||
docker push registry.gitlab.com/itsulu-odoo/itsulu-blog-publisher:latest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. GitLab CI Pipeline
|
||||
|
||||
```yaml
|
||||
# .gitlab-ci.yml (skeleton)
|
||||
|
|
@ -507,7 +636,7 @@ Never put secrets in `.gitlab-ci.yml`. Never `echo $SECRET` in scripts.
|
|||
|
||||
---
|
||||
|
||||
## 9. Pre-commit Configuration
|
||||
## 10. Pre-commit Configuration
|
||||
|
||||
```yaml
|
||||
# .pre-commit-config.yaml
|
||||
|
|
@ -538,7 +667,7 @@ Run on commit: automatic after `pre-commit install`
|
|||
|
||||
---
|
||||
|
||||
## 10. CLAUDE.md Template
|
||||
## 11. CLAUDE.md Template
|
||||
|
||||
Copy this to the repo root and fill in the blanks. Claude Code reads this automatically.
|
||||
|
||||
|
|
@ -604,7 +733,7 @@ Show options with trade-offs. I will choose.
|
|||
|
||||
---
|
||||
|
||||
## 11. Failure Recovery Quick Reference
|
||||
## 12. Failure Recovery Quick Reference
|
||||
|
||||
| Symptom | Most likely cause | Fix |
|
||||
|---|---|---|
|
||||
|
|
@ -615,10 +744,17 @@ Show options with trade-offs. I will choose.
|
|||
| Flaky test fails > twice/week | Non-deterministic test | Quarantine with `@pytest.mark.skip(reason="flaky #123")`; fix within 1 week |
|
||||
| Secret appears in CI logs | Variable not masked | Rotate immediately; add to masked variables in GitLab |
|
||||
| ArgoCD stuck OutOfSync | Manual cluster change | `argocd app diff`; revert manual change; re-sync |
|
||||
| K8s Job: `PermissionError: __manifest__.py` | Files copied without `--chown=odoo:odoo` | Rebuild image with `COPY --chown=odoo:odoo` in Dockerfile |
|
||||
| K8s Job: `No module named pytest` | Ran as wrong user (root); pytest in odoo user's ~/.local | Run container as odoo user (default); use `python3 -m pytest` not `pytest` |
|
||||
| K8s Job: `database does not exist` | Init container used `$RANDOM` — different from test-runner | Use fixed DB name (`odoo_test`) shared between containers |
|
||||
| K8s Job: `ErrImagePull` | `gitlab-docker-creds` secret missing or expired | Recreate: `kubectl create secret docker-registry gitlab-docker-creds ...` |
|
||||
| K8s Job: `CreateContainerConfigError` | Secret key missing (e.g. `database` key not in `test-db-info`) | `kubectl describe pod <pod>` to find missing key; recreate secret with all 3 keys |
|
||||
| Docker build: `chmod: Operation not permitted` | odoo:17.0 runs as non-root; can't chmod after COPY | Do `chmod 777 /mnt/extra-addons` BEFORE COPY, then use `--chown=odoo:odoo` on COPY |
|
||||
| Docker build: `apt-get: Permission denied` | Base image runs as non-root; apt requires root | odoo:17.0 has no apt access — use `python3 -m pip` instead; no system packages possible |
|
||||
|
||||
---
|
||||
|
||||
## 12. SLO Targets
|
||||
## 13. SLO Targets
|
||||
|
||||
| Metric | Target | Why |
|
||||
|---|---|---|
|
||||
|
|
@ -632,7 +768,7 @@ Show options with trade-offs. I will choose.
|
|||
|
||||
---
|
||||
|
||||
## 13. Phased Adoption Triggers
|
||||
## 14. Phased Adoption Triggers
|
||||
|
||||
**Add Phase 2 (BDD, factories, coverage gates) when ≥ 2:**
|
||||
- Stakeholders ask "how do we know X works?"
|
||||
|
|
|
|||
14
Dockerfile
14
Dockerfile
|
|
@ -1,8 +1,7 @@
|
|||
FROM odoo:17.0
|
||||
|
||||
# Install Python testing dependencies directly with pip
|
||||
# (Odoo base image already has system dependencies)
|
||||
RUN pip install --no-cache-dir \
|
||||
# Install Python testing dependencies using the system Python
|
||||
RUN python3 -m pip install --no-cache-dir \
|
||||
pytest \
|
||||
pytest-odoo \
|
||||
pytest-bdd \
|
||||
|
|
@ -11,10 +10,11 @@ RUN pip install --no-cache-dir \
|
|||
requests
|
||||
|
||||
# Copy addon to Odoo addons path
|
||||
COPY addons/itsulu_blog_publisher /mnt/extra-addons/itsulu_blog_publisher
|
||||
RUN mkdir -p /mnt/extra-addons && chmod 777 /mnt/extra-addons
|
||||
COPY --chown=odoo:odoo addons/itsulu_blog_publisher /mnt/extra-addons/itsulu_blog_publisher
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /mnt/extra-addons/itsulu_blog_publisher
|
||||
# Set working directory to a temp location for test running
|
||||
WORKDIR /tmp/test
|
||||
|
||||
# Default command runs tests (can be overridden)
|
||||
CMD ["pytest", "tests/", "-v", "--html=/tmp/report.html", "--self-contained-html"]
|
||||
CMD ["python3", "-m", "pytest", "tests/", "-v", "--html=/tmp/report.html", "--self-contained-html"]
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
{
|
||||
'name': 'ITSulu Blog Publisher',
|
||||
'version': '14.0.1.0.0',
|
||||
'version': '17.0.1.0.0',
|
||||
'summary': 'AI-powered blog post generation with multi-LLM support, scheduling, and social media copy',
|
||||
'description': """
|
||||
ITSulu Blog Publisher
|
||||
|
|
@ -39,8 +39,6 @@ Features
|
|||
'security/ir.model.access.csv',
|
||||
'data/mail_template_data.xml',
|
||||
'data/ir_cron_data.xml',
|
||||
'data/default_prompts_data.xml',
|
||||
'views/menu_views.xml',
|
||||
'views/blog_topic_views.xml',
|
||||
'views/blog_schedule_views.xml',
|
||||
'views/blog_generation_log_views.xml',
|
||||
|
|
@ -48,6 +46,7 @@ Features
|
|||
'views/generate_now_wizard_views.xml',
|
||||
'views/res_config_settings_views.xml',
|
||||
'views/website_blog_publisher_templates.xml',
|
||||
'views/menu_views.xml',
|
||||
],
|
||||
'assets': {
|
||||
'web.assets_backend': [
|
||||
|
|
|
|||
|
|
@ -64,8 +64,7 @@ class BlogSchedule(models.Model):
|
|||
blog_id = fields.Many2one(
|
||||
comodel_name='blog.blog',
|
||||
string='Target Blog',
|
||||
required=True,
|
||||
help='Blog where generated posts will be created.',
|
||||
help='Blog where generated posts will be created. Must be set before activating this slot.',
|
||||
)
|
||||
|
||||
tone = fields.Char(
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@
|
|||
<field name="duration_seconds" string="Dur (s)" optional="show"/>
|
||||
<field name="blog_post_id"/>
|
||||
<button name="action_retry" type="object" string="↩ Retry"
|
||||
attrs="{'invisible': [('state', '!=', 'error')]}"
|
||||
invisible="state != 'error'"
|
||||
class="btn-sm btn-warning"
|
||||
data-test-id="btn-retry-generation"/>
|
||||
</tree>
|
||||
|
|
@ -41,7 +41,7 @@
|
|||
<form string="Generation Log" create="false" edit="false">
|
||||
<header>
|
||||
<button name="action_retry" type="object" string="↩ Retry Generation"
|
||||
attrs="{'invisible': [('state', '!=', 'error')]}"
|
||||
invisible="state != 'error'"
|
||||
class="btn-warning"
|
||||
data-test-id="btn-retry-log-form"/>
|
||||
</header>
|
||||
|
|
@ -50,7 +50,7 @@
|
|||
<group string="Result">
|
||||
<field name="state" widget="badge"/>
|
||||
<field name="blog_post_id"/>
|
||||
<field name="error_message" attrs="{'invisible': [('state', '!=', 'error')]}"/>
|
||||
<field name="error_message" invisible="state != 'error'"/>
|
||||
</group>
|
||||
<group string="Trigger">
|
||||
<field name="trigger_source"/>
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@
|
|||
</header>
|
||||
<sheet>
|
||||
<widget name="web_ribbon" title="Inactive" bg_color="bg-danger"
|
||||
attrs="{'invisible': [('active', '=', True)]}"/>
|
||||
invisible="active"/>
|
||||
<div class="oe_title">
|
||||
<h1><field name="name" placeholder="e.g. Morning Post"/></h1>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -20,10 +20,10 @@
|
|||
decoration-warning="state=='skipped'"/>
|
||||
<field name="used_date" optional="show"/>
|
||||
<button name="action_mark_pending" type="object" string="↩ Reset"
|
||||
attrs="{'invisible': [('state', '=', 'pending')]}"
|
||||
invisible="state == 'pending'"
|
||||
class="btn-sm btn-secondary"/>
|
||||
<button name="action_mark_skipped" type="object" string="Skip"
|
||||
attrs="{'invisible': [('state', '!=', 'pending')]}"
|
||||
invisible="state != 'pending'"
|
||||
class="btn-sm btn-warning"/>
|
||||
</tree>
|
||||
</field>
|
||||
|
|
@ -36,10 +36,10 @@
|
|||
<form string="Blog Topic">
|
||||
<header>
|
||||
<button name="action_mark_pending" type="object" string="Reset to Pending"
|
||||
attrs="{'invisible': [('state', '=', 'pending')]}"
|
||||
invisible="state == 'pending'"
|
||||
class="btn-secondary"/>
|
||||
<button name="action_mark_skipped" type="object" string="Skip"
|
||||
attrs="{'invisible': [('state', '!=', 'pending')]}"
|
||||
invisible="state != 'pending'"
|
||||
class="btn-warning"/>
|
||||
<field name="state" widget="statusbar" statusbar_visible="pending,used"/>
|
||||
</header>
|
||||
|
|
|
|||
Loading…
Reference in a new issue