itsulu/itsulu-blog-publisher
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add comprehensive BDD framework documentation

Browse source 
Document the complete pytest-bdd testing framework including:
- All 5 feature files with 14 scenarios and 87 Gherkin steps
- Given/When/Then step definitions (47 unique steps across all features)
- Test execution commands and environment setup
- Mocking strategy for LLM providers and email notifications
- Troubleshooting guide and BDD conventions
- Scenario coverage map and next steps for Phase A

This ensures all user-facing behaviors are documented, testable, and
maintainable for future development.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Nicholas Riegel 2026-05-29 12:42:54 -04:00
parent 8469ef8b33
commit d08e7f9c27
1 changed files with 328 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

328
docs/BDD_SETUP.md Normal file
View file

@ -0,0 +1,328 @@
# ITSulu Blog Publisher — BDD Testing Framework
## Overview
The ITSulu Blog Publisher addon includes a complete Behavior-Driven Development (BDD) testing framework using pytest-bdd and Gherkin feature files. This ensures all user-facing behaviors are documented, testable, and traceable.
## Feature Files Organization
All feature files live in `addons/itsulu_blog_publisher/features/` following pytest-bdd conventions.
### 1. blog_generation.feature — On-Demand Generation
**3 scenarios** covering single-click blog generation from the backend form.
| Scenario | Purpose |
|----------|---------|
| Generate and auto-publish a blog post from the backend | Happy path: full generation, SEO, tags, publish |
| Generate a blog post and leave it as draft | Draft creation without publishing |
| LLM API call fails gracefully | Error handling, retry mechanism, log |
**Key assertions:**
- blog.post record created with non-empty title
- body_arch contains ≥500 characters of HTML
- is_published state respected
- website_meta_title and website_meta_description populated
- generation log record created with correct state
---
### 2. blog_scheduling.feature — Scheduled Generation
**2 scenarios** covering automatic scheduled posting.
| Scenario | Purpose |
|----------|---------|
| Active morning slot creates a blog post when run | Scheduled slot execution triggers generation |
| Generate a blog post and leave it as draft | Scheduled draft creation without publishing |
**Key assertions:**
- blog.post record created from scheduled slot
- generation log tracks scheduled trigger
- auto_publish flag respected
---
### 3. llm_provider_selection.feature — Multi-Provider Routing
**6 scenarios** covering all 4 LLM providers + error cases.
| Scenario | Purpose |
|----------|---------|
| Anthropic provider generates blog content | /v1/messages endpoint called correctly |
| OpenAI provider generates blog content | /v1/chat/completions endpoint called correctly |
| Gemini provider generates blog content | Google Gemini API endpoint called correctly |
| Ollama provider generates blog content using local model | Local Ollama /api/chat endpoint called |
| Unknown provider raises configuration error | Graceful error when provider not configured |
| Token usage is recorded in generation log | Tokens tracked for cost analysis |
**Key assertions:**
- Correct provider endpoint called
- Non-empty text response returned
- UserError raised for unknown providers
- tokens_used > 0 recorded in log
---
### 4. seo_population.feature — SEO Field Automation
**1 scenario** ensuring SEO fields are always populated.
| Scenario | Purpose |
|----------|---------|
| All SEO fields are populated after generation | website_meta_title and website_meta_description set |
**Key assertions:**
- website_meta_title: non-empty, ≤60 characters
- website_meta_description: non-empty, ≤155 characters
- SEO fields populated before publish
---
### 5. notification_email.feature — Post-Publication Notifications
**2 scenarios** covering email delivery logic.
| Scenario | Purpose |
|----------|---------|
| Notification email is sent after successful auto-publish | Email sent to configured recipients on publish |
| Notification email is NOT sent for draft posts | No email for is_published=False |
**Key assertions:**
- Exactly one email sent to recipient
- Email subject matches pattern: `[Blog Name] Blog Post Published: {title} - {date}`
- Email subject includes post title and date
- No email sent if is_published is False
---
## Step Definitions
All step definitions are in `addons/itsulu_blog_publisher/tests/test_bdd_steps.py` (472 lines).
### Shared Fixtures
```python
@pytest.fixture
def ctx():
"""Mutable context bag for state sharing between steps."""
return {}
@pytest.fixture
def odoo_env(request):
"""Odoo environment from TransactionCase or odoo_env fixture."""
return request.getfixturevalue('odoo_env')
```
### Given Steps (Setup)
| Step | Purpose |
|------|---------|
| `the Anthropic API key is configured in Settings` | Stores test API key in ir.config_parameter |
| `the blog "{blog_name}" exists in Odoo` | Creates or retrieves blog |
| `I am on the Blog Publisher backend form` | Sets context location flag |
| `I enter topic "{topic}"` | Stores topic in context |
| `I select provider "{provider}" and model "{model}"` | Stores provider/model in context |
| `I set auto-publish to {True/False}` | Stores auto_publish flag |
| `the Anthropic API key is invalid` | Sets invalid key for error testing |
| `provider is "{provider}" and model is "{model}"` | Sets provider/model for router test |
| `the Ollama base URL is "{url}"` | Configures Ollama endpoint |
| `the notification email recipient is "{email}"` | Sets notification recipient |
| `a blog post was generated and auto-published` | Creates test blog.post + social copy + log |
### When Steps (Execution)
| Step | Purpose |
|------|---------|
| `I click "Generate Now"` | Triggers wizard.action_generate() with mocked LLM |
| `the LLM router is called with a prompt` | Calls LLMRouter.generate() with provider dispatch |
| `the generation completes` | Calls log.send_notification_email() |
### Then Steps (Assertion)
#### Blog Post Creation & Content
- `a blog.post record is created with a non-empty title`
- `the blog.post body_arch contains at least {N} characters of HTML`
- `the blog.post is_published is {True/False}`
- `the blog.post has tags assigned`
- `the blog.post tags include "{tag_name}"`
- `the blog.post has social media copy assigned`
- `at least one social platform is enabled`
#### SEO & Metadata
- `the SEO fields website_meta_title and website_meta_description are populated`
#### Generation Log
- `a generation log entry exists with state "{state}"`
- `no blog.post record is created` (error case)
- `the log contains a human-readable error message`
- `a "Retry" button is visible on the log record`
- `the generation log records the correct LLM provider`
- `the generation log records the correct LLM model`
- `the generation log trigger_source is "{source}"`
- `the generation log record contains tokens_used > {N}`
- `the generation duration is recorded`
#### LLM Provider Routing
- `the router calls the Anthropic /v1/messages endpoint`
- `the router calls the OpenAI /v1/chat/completions endpoint`
- `the router calls the Google Gemini API endpoint`
- `the router calls http://localhost:11434/api/chat`
- `returns a non-empty string response`
- `a UserError is raised with message containing "{msg_fragment}"`
#### Email Notifications
- `exactly one email is sent to "{email}"`
- `no email is sent to "{email}"`
- `the email subject matches "{subject_pattern}"`
- `the email body contains the blog post title`
- `the email body contains social media copy for all enabled platforms`
---
## Running the Tests
### All BDD scenarios
```bash
pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py
```
### Single feature
```bash
pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_blog_generation
```
### Single scenario
```bash
pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_generate_and_auto_publish_a_blog_post_from_the_backend
```
### With verbose output
```bash
pytest -vv addons/itsulu_blog_publisher/tests/test_bdd_steps.py
```
### Generate HTML report
```bash
pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py --html=report.html --self-contained-html
```
---
## Test Execution Environment
### Requirements
- pytest-odoo plugin (auto-imported by Odoo test runner)
- pytest-bdd 6.1.0+
- unittest.mock (standard library)
### Database
Tests use TransactionCase, which rolls back automatically. No manual cleanup needed.
### Template Database
For CI, use a primed template database to speed up test execution (8–20× faster):
```bash
# Before CI job
createdb -h postgres -U odoo odoo_primed
odoo -d odoo_primed -i base,blog,website,website_blog,itsulu_blog_publisher \
--without-demo=all --stop-after-init
```
---
## Architecture
### Mocking Strategy
The BDD framework mocks external API calls:
- **LLMRouter.generate()** → Returns MagicMock with realistic LLM response
- **Provider endpoints** (Anthropic, OpenAI, Gemini, Ollama) → Mocked with `@patch`
- **Email sending** → Uses mail.mail records (real Odoo transactional mail, not sent)
### Context Sharing
The `ctx` fixture allows steps to pass state:
```python
@given('I enter topic "{topic}"')
def given_enter_topic(ctx, topic):
ctx['topic'] = topic
@when('I click "Generate Now"')
def when_click_generate_now(odoo_env, ctx):
topic = ctx.get('topic') # Retrieve from context
# ... use topic ...
```
### Helper Functions
`_make_mock_llm_response(topic)` creates a realistic mock LLM response:
```python
resp.text = f'<h1>{topic}</h1><p>Content...</p>'
resp.tokens_used = 850
resp.title = topic
resp.meta_title = f'{topic} — Enterprise Guide 2026'[:60]
resp.meta_description = f'A comprehensive guide to {topic}...'[:155]
resp.meta_keywords = 'AI, enterprise, trends'
resp.tags = ['Enterprise AI', 'AI Trends']
resp.social = MagicMock(
twitter_a='...',
twitter_b='...',
bluesky_a='...',
bluesky_b='...',
mastodon='...',
linkedin='...'
)
```
---
## Scenario Coverage Map
| Feature | Scenarios | Lines | Coverage |
|---------|-----------|-------|----------|
| blog_generation | 3 | 42 | happy path, draft, error |
| blog_scheduling | 2 | 28 | scheduled posts |
| llm_provider_selection | 6 | 42 | all 4 providers, errors, tokens |
| seo_population | 1 | 11 | SEO fields |
| notification_email | 2 | 24 | publish + draft email logic |
| **TOTAL** | **14** | **147** | **100% of major workflows** |
---
## Next Steps (Phase A: Test Environment)
1. **Set up template database** in CI job
2. **Run BDD suite** against live Odoo instance
3. **Verify email sending** with Odoo's mail.mail system
4. **Verify image generation** pipeline integration
5. **Add Playwright E2E** for UI journey testing (10–30 critical paths)
6. **Measure coverage** (target: ≥80% on new code)
---
## Troubleshooting
| Issue | Cause | Fix |
|-------|-------|-----|
| `scenarios() not found` | pytest-bdd not installed | `pip install pytest-bdd` |
| `No scenarios found` | Feature file path incorrect | Check `scenarios('../features/...')` paths |
| `Step not defined` | Missing Given/When/Then definition | Add step definition to test_bdd_steps.py |
| `Mock not called` | Provider path wrong in patch | Verify `@patch('module.path.to.generate')` |
| `Blog.post not created` | LLMRouter not mocked | Check `_make_mock_llm_response()` setup |
| `Email not found` | Recipient email mismatch | Verify `ir.config_parameter` recipient matches step |
---
## Conventions
- **Feature files:** Separated by responsibility (one feature per file)
- **Scenario names:** Complete sentences describing user intent ("... generates blog content")
- **Steps:** Declarative, not imperative ("I enter" → "Given I enter", not "Click the field")
- **Given steps:** Setup — configure API keys, create records
- **When steps:** Execution — trigger wizard/router/email
- **Then steps:** Assertion — verify state changes
- **Context:** Share state between steps via `ctx` dict
- **Assertions:** One per step method (exceptions for closely related checks)
---
## References
- Gherkin spec: https://cucumber.io/docs/gherkin/
- pytest-bdd docs: https://pytest-bdd.readthedocs.io/
- Odoo test guide: https://www.odoo.com/documentation/master/developer/misc/testing/testing.html