diff --git a/docs/BDD_SETUP.md b/docs/BDD_SETUP.md new file mode 100644 index 0000000..4e88a29 --- /dev/null +++ b/docs/BDD_SETUP.md @@ -0,0 +1,328 @@ +# ITSulu Blog Publisher — BDD Testing Framework + +## Overview + +The ITSulu Blog Publisher addon includes a complete Behavior-Driven Development (BDD) testing framework using pytest-bdd and Gherkin feature files. This ensures all user-facing behaviors are documented, testable, and traceable. + +## Feature Files Organization + +All feature files live in `addons/itsulu_blog_publisher/features/` following pytest-bdd conventions. + +### 1. blog_generation.feature — On-Demand Generation +**3 scenarios** covering single-click blog generation from the backend form. + +| Scenario | Purpose | +|----------|---------| +| Generate and auto-publish a blog post from the backend | Happy path: full generation, SEO, tags, publish | +| Generate a blog post and leave it as draft | Draft creation without publishing | +| LLM API call fails gracefully | Error handling, retry mechanism, log | + +**Key assertions:** +- blog.post record created with non-empty title +- body_arch contains ≥500 characters of HTML +- is_published state respected +- website_meta_title and website_meta_description populated +- generation log record created with correct state + +--- + +### 2. blog_scheduling.feature — Scheduled Generation +**2 scenarios** covering automatic scheduled posting. + +| Scenario | Purpose | +|----------|---------| +| Active morning slot creates a blog post when run | Scheduled slot execution triggers generation | +| Generate a blog post and leave it as draft | Scheduled draft creation without publishing | + +**Key assertions:** +- blog.post record created from scheduled slot +- generation log tracks scheduled trigger +- auto_publish flag respected + +--- + +### 3. llm_provider_selection.feature — Multi-Provider Routing +**6 scenarios** covering all 4 LLM providers + error cases. + +| Scenario | Purpose | +|----------|---------| +| Anthropic provider generates blog content | /v1/messages endpoint called correctly | +| OpenAI provider generates blog content | /v1/chat/completions endpoint called correctly | +| Gemini provider generates blog content | Google Gemini API endpoint called correctly | +| Ollama provider generates blog content using local model | Local Ollama /api/chat endpoint called | +| Unknown provider raises configuration error | Graceful error when provider not configured | +| Token usage is recorded in generation log | Tokens tracked for cost analysis | + +**Key assertions:** +- Correct provider endpoint called +- Non-empty text response returned +- UserError raised for unknown providers +- tokens_used > 0 recorded in log + +--- + +### 4. seo_population.feature — SEO Field Automation +**1 scenario** ensuring SEO fields are always populated. + +| Scenario | Purpose | +|----------|---------| +| All SEO fields are populated after generation | website_meta_title and website_meta_description set | + +**Key assertions:** +- website_meta_title: non-empty, ≤60 characters +- website_meta_description: non-empty, ≤155 characters +- SEO fields populated before publish + +--- + +### 5. notification_email.feature — Post-Publication Notifications +**2 scenarios** covering email delivery logic. + +| Scenario | Purpose | +|----------|---------| +| Notification email is sent after successful auto-publish | Email sent to configured recipients on publish | +| Notification email is NOT sent for draft posts | No email for is_published=False | + +**Key assertions:** +- Exactly one email sent to recipient +- Email subject matches pattern: `[Blog Name] Blog Post Published: {title} - {date}` +- Email subject includes post title and date +- No email sent if is_published is False + +--- + +## Step Definitions + +All step definitions are in `addons/itsulu_blog_publisher/tests/test_bdd_steps.py` (472 lines). + +### Shared Fixtures + +```python +@pytest.fixture +def ctx(): + """Mutable context bag for state sharing between steps.""" + return {} + +@pytest.fixture +def odoo_env(request): + """Odoo environment from TransactionCase or odoo_env fixture.""" + return request.getfixturevalue('odoo_env') +``` + +### Given Steps (Setup) + +| Step | Purpose | +|------|---------| +| `the Anthropic API key is configured in Settings` | Stores test API key in ir.config_parameter | +| `the blog "{blog_name}" exists in Odoo` | Creates or retrieves blog | +| `I am on the Blog Publisher backend form` | Sets context location flag | +| `I enter topic "{topic}"` | Stores topic in context | +| `I select provider "{provider}" and model "{model}"` | Stores provider/model in context | +| `I set auto-publish to {True/False}` | Stores auto_publish flag | +| `the Anthropic API key is invalid` | Sets invalid key for error testing | +| `provider is "{provider}" and model is "{model}"` | Sets provider/model for router test | +| `the Ollama base URL is "{url}"` | Configures Ollama endpoint | +| `the notification email recipient is "{email}"` | Sets notification recipient | +| `a blog post was generated and auto-published` | Creates test blog.post + social copy + log | + +### When Steps (Execution) + +| Step | Purpose | +|------|---------| +| `I click "Generate Now"` | Triggers wizard.action_generate() with mocked LLM | +| `the LLM router is called with a prompt` | Calls LLMRouter.generate() with provider dispatch | +| `the generation completes` | Calls log.send_notification_email() | + +### Then Steps (Assertion) + +#### Blog Post Creation & Content +- `a blog.post record is created with a non-empty title` +- `the blog.post body_arch contains at least {N} characters of HTML` +- `the blog.post is_published is {True/False}` +- `the blog.post has tags assigned` +- `the blog.post tags include "{tag_name}"` +- `the blog.post has social media copy assigned` +- `at least one social platform is enabled` + +#### SEO & Metadata +- `the SEO fields website_meta_title and website_meta_description are populated` + +#### Generation Log +- `a generation log entry exists with state "{state}"` +- `no blog.post record is created` (error case) +- `the log contains a human-readable error message` +- `a "Retry" button is visible on the log record` +- `the generation log records the correct LLM provider` +- `the generation log records the correct LLM model` +- `the generation log trigger_source is "{source}"` +- `the generation log record contains tokens_used > {N}` +- `the generation duration is recorded` + +#### LLM Provider Routing +- `the router calls the Anthropic /v1/messages endpoint` +- `the router calls the OpenAI /v1/chat/completions endpoint` +- `the router calls the Google Gemini API endpoint` +- `the router calls http://localhost:11434/api/chat` +- `returns a non-empty string response` +- `a UserError is raised with message containing "{msg_fragment}"` + +#### Email Notifications +- `exactly one email is sent to "{email}"` +- `no email is sent to "{email}"` +- `the email subject matches "{subject_pattern}"` +- `the email body contains the blog post title` +- `the email body contains social media copy for all enabled platforms` + +--- + +## Running the Tests + +### All BDD scenarios +```bash +pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py +``` + +### Single feature +```bash +pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_blog_generation +``` + +### Single scenario +```bash +pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_generate_and_auto_publish_a_blog_post_from_the_backend +``` + +### With verbose output +```bash +pytest -vv addons/itsulu_blog_publisher/tests/test_bdd_steps.py +``` + +### Generate HTML report +```bash +pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py --html=report.html --self-contained-html +``` + +--- + +## Test Execution Environment + +### Requirements +- pytest-odoo plugin (auto-imported by Odoo test runner) +- pytest-bdd 6.1.0+ +- unittest.mock (standard library) + +### Database +Tests use TransactionCase, which rolls back automatically. No manual cleanup needed. + +### Template Database +For CI, use a primed template database to speed up test execution (8–20× faster): +```bash +# Before CI job +createdb -h postgres -U odoo odoo_primed +odoo -d odoo_primed -i base,blog,website,website_blog,itsulu_blog_publisher \ + --without-demo=all --stop-after-init +``` + +--- + +## Architecture + +### Mocking Strategy + +The BDD framework mocks external API calls: +- **LLMRouter.generate()** → Returns MagicMock with realistic LLM response +- **Provider endpoints** (Anthropic, OpenAI, Gemini, Ollama) → Mocked with `@patch` +- **Email sending** → Uses mail.mail records (real Odoo transactional mail, not sent) + +### Context Sharing + +The `ctx` fixture allows steps to pass state: +```python +@given('I enter topic "{topic}"') +def given_enter_topic(ctx, topic): + ctx['topic'] = topic + +@when('I click "Generate Now"') +def when_click_generate_now(odoo_env, ctx): + topic = ctx.get('topic') # Retrieve from context + # ... use topic ... +``` + +### Helper Functions + +`_make_mock_llm_response(topic)` creates a realistic mock LLM response: +```python +resp.text = f'

{topic}

Content...

' +resp.tokens_used = 850 +resp.title = topic +resp.meta_title = f'{topic} — Enterprise Guide 2026'[:60] +resp.meta_description = f'A comprehensive guide to {topic}...'[:155] +resp.meta_keywords = 'AI, enterprise, trends' +resp.tags = ['Enterprise AI', 'AI Trends'] +resp.social = MagicMock( + twitter_a='...', + twitter_b='...', + bluesky_a='...', + bluesky_b='...', + mastodon='...', + linkedin='...' +) +``` + +--- + +## Scenario Coverage Map + +| Feature | Scenarios | Lines | Coverage | +|---------|-----------|-------|----------| +| blog_generation | 3 | 42 | happy path, draft, error | +| blog_scheduling | 2 | 28 | scheduled posts | +| llm_provider_selection | 6 | 42 | all 4 providers, errors, tokens | +| seo_population | 1 | 11 | SEO fields | +| notification_email | 2 | 24 | publish + draft email logic | +| **TOTAL** | **14** | **147** | **100% of major workflows** | + +--- + +## Next Steps (Phase A: Test Environment) + +1. **Set up template database** in CI job +2. **Run BDD suite** against live Odoo instance +3. **Verify email sending** with Odoo's mail.mail system +4. **Verify image generation** pipeline integration +5. **Add Playwright E2E** for UI journey testing (10–30 critical paths) +6. **Measure coverage** (target: ≥80% on new code) + +--- + +## Troubleshooting + +| Issue | Cause | Fix | +|-------|-------|-----| +| `scenarios() not found` | pytest-bdd not installed | `pip install pytest-bdd` | +| `No scenarios found` | Feature file path incorrect | Check `scenarios('../features/...')` paths | +| `Step not defined` | Missing Given/When/Then definition | Add step definition to test_bdd_steps.py | +| `Mock not called` | Provider path wrong in patch | Verify `@patch('module.path.to.generate')` | +| `Blog.post not created` | LLMRouter not mocked | Check `_make_mock_llm_response()` setup | +| `Email not found` | Recipient email mismatch | Verify `ir.config_parameter` recipient matches step | + +--- + +## Conventions + +- **Feature files:** Separated by responsibility (one feature per file) +- **Scenario names:** Complete sentences describing user intent ("... generates blog content") +- **Steps:** Declarative, not imperative ("I enter" → "Given I enter", not "Click the field") +- **Given steps:** Setup — configure API keys, create records +- **When steps:** Execution — trigger wizard/router/email +- **Then steps:** Assertion — verify state changes +- **Context:** Share state between steps via `ctx` dict +- **Assertions:** One per step method (exceptions for closely related checks) + +--- + +## References + +- Gherkin spec: https://cucumber.io/docs/gherkin/ +- pytest-bdd docs: https://pytest-bdd.readthedocs.io/ +- Odoo test guide: https://www.odoo.com/documentation/master/developer/misc/testing/testing.html