itsulu-blog-publisher/docs/BDD_SETUP.md
Nicholas Riegel d08e7f9c27 Add comprehensive BDD framework documentation
Document the complete pytest-bdd testing framework including:
- All 5 feature files with 14 scenarios and 87 Gherkin steps
- Given/When/Then step definitions (47 unique steps across all features)
- Test execution commands and environment setup
- Mocking strategy for LLM providers and email notifications
- Troubleshooting guide and BDD conventions
- Scenario coverage map and next steps for Phase A

This ensures all user-facing behaviors are documented, testable, and
maintainable for future development.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-05-29 12:42:54 -04:00

11 KiB
Raw Permalink Blame History

ITSulu Blog Publisher — BDD Testing Framework

Overview

The ITSulu Blog Publisher addon includes a complete Behavior-Driven Development (BDD) testing framework using pytest-bdd and Gherkin feature files. This ensures all user-facing behaviors are documented, testable, and traceable.

Feature Files Organization

All feature files live in addons/itsulu_blog_publisher/features/ following pytest-bdd conventions.

1. blog_generation.feature — On-Demand Generation

3 scenarios covering single-click blog generation from the backend form.

Scenario Purpose
Generate and auto-publish a blog post from the backend Happy path: full generation, SEO, tags, publish
Generate a blog post and leave it as draft Draft creation without publishing
LLM API call fails gracefully Error handling, retry mechanism, log

Key assertions:

  • blog.post record created with non-empty title
  • body_arch contains ≥500 characters of HTML
  • is_published state respected
  • website_meta_title and website_meta_description populated
  • generation log record created with correct state

2. blog_scheduling.feature — Scheduled Generation

2 scenarios covering automatic scheduled posting.

Scenario Purpose
Active morning slot creates a blog post when run Scheduled slot execution triggers generation
Generate a blog post and leave it as draft Scheduled draft creation without publishing

Key assertions:

  • blog.post record created from scheduled slot
  • generation log tracks scheduled trigger
  • auto_publish flag respected

3. llm_provider_selection.feature — Multi-Provider Routing

6 scenarios covering all 4 LLM providers + error cases.

Scenario Purpose
Anthropic provider generates blog content /v1/messages endpoint called correctly
OpenAI provider generates blog content /v1/chat/completions endpoint called correctly
Gemini provider generates blog content Google Gemini API endpoint called correctly
Ollama provider generates blog content using local model Local Ollama /api/chat endpoint called
Unknown provider raises configuration error Graceful error when provider not configured
Token usage is recorded in generation log Tokens tracked for cost analysis

Key assertions:

  • Correct provider endpoint called
  • Non-empty text response returned
  • UserError raised for unknown providers
  • tokens_used > 0 recorded in log

4. seo_population.feature — SEO Field Automation

1 scenario ensuring SEO fields are always populated.

Scenario Purpose
All SEO fields are populated after generation website_meta_title and website_meta_description set

Key assertions:

  • website_meta_title: non-empty, ≤60 characters
  • website_meta_description: non-empty, ≤155 characters
  • SEO fields populated before publish

5. notification_email.feature — Post-Publication Notifications

2 scenarios covering email delivery logic.

Scenario Purpose
Notification email is sent after successful auto-publish Email sent to configured recipients on publish
Notification email is NOT sent for draft posts No email for is_published=False

Key assertions:

  • Exactly one email sent to recipient
  • Email subject matches pattern: [Blog Name] Blog Post Published: {title} - {date}
  • Email subject includes post title and date
  • No email sent if is_published is False

Step Definitions

All step definitions are in addons/itsulu_blog_publisher/tests/test_bdd_steps.py (472 lines).

Shared Fixtures

@pytest.fixture
def ctx():
    """Mutable context bag for state sharing between steps."""
    return {}

@pytest.fixture
def odoo_env(request):
    """Odoo environment from TransactionCase or odoo_env fixture."""
    return request.getfixturevalue('odoo_env')

Given Steps (Setup)

Step Purpose
the Anthropic API key is configured in Settings Stores test API key in ir.config_parameter
the blog "{blog_name}" exists in Odoo Creates or retrieves blog
I am on the Blog Publisher backend form Sets context location flag
I enter topic "{topic}" Stores topic in context
I select provider "{provider}" and model "{model}" Stores provider/model in context
I set auto-publish to {True/False} Stores auto_publish flag
the Anthropic API key is invalid Sets invalid key for error testing
provider is "{provider}" and model is "{model}" Sets provider/model for router test
the Ollama base URL is "{url}" Configures Ollama endpoint
the notification email recipient is "{email}" Sets notification recipient
a blog post was generated and auto-published Creates test blog.post + social copy + log

When Steps (Execution)

Step Purpose
I click "Generate Now" Triggers wizard.action_generate() with mocked LLM
the LLM router is called with a prompt Calls LLMRouter.generate() with provider dispatch
the generation completes Calls log.send_notification_email()

Then Steps (Assertion)

Blog Post Creation & Content

  • a blog.post record is created with a non-empty title
  • the blog.post body_arch contains at least {N} characters of HTML
  • the blog.post is_published is {True/False}
  • the blog.post has tags assigned
  • the blog.post tags include "{tag_name}"
  • the blog.post has social media copy assigned
  • at least one social platform is enabled

SEO & Metadata

  • the SEO fields website_meta_title and website_meta_description are populated

Generation Log

  • a generation log entry exists with state "{state}"
  • no blog.post record is created (error case)
  • the log contains a human-readable error message
  • a "Retry" button is visible on the log record
  • the generation log records the correct LLM provider
  • the generation log records the correct LLM model
  • the generation log trigger_source is "{source}"
  • the generation log record contains tokens_used > {N}
  • the generation duration is recorded

LLM Provider Routing

  • the router calls the Anthropic /v1/messages endpoint
  • the router calls the OpenAI /v1/chat/completions endpoint
  • the router calls the Google Gemini API endpoint
  • the router calls http://localhost:11434/api/chat
  • returns a non-empty string response
  • a UserError is raised with message containing "{msg_fragment}"

Email Notifications

  • exactly one email is sent to "{email}"
  • no email is sent to "{email}"
  • the email subject matches "{subject_pattern}"
  • the email body contains the blog post title
  • the email body contains social media copy for all enabled platforms

Running the Tests

All BDD scenarios

pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py

Single feature

pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_blog_generation

Single scenario

pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py::test_generate_and_auto_publish_a_blog_post_from_the_backend

With verbose output

pytest -vv addons/itsulu_blog_publisher/tests/test_bdd_steps.py

Generate HTML report

pytest addons/itsulu_blog_publisher/tests/test_bdd_steps.py --html=report.html --self-contained-html

Test Execution Environment

Requirements

  • pytest-odoo plugin (auto-imported by Odoo test runner)
  • pytest-bdd 6.1.0+
  • unittest.mock (standard library)

Database

Tests use TransactionCase, which rolls back automatically. No manual cleanup needed.

Template Database

For CI, use a primed template database to speed up test execution (820× faster):

# Before CI job
createdb -h postgres -U odoo odoo_primed
odoo -d odoo_primed -i base,blog,website,website_blog,itsulu_blog_publisher \
  --without-demo=all --stop-after-init

Architecture

Mocking Strategy

The BDD framework mocks external API calls:

  • LLMRouter.generate() → Returns MagicMock with realistic LLM response
  • Provider endpoints (Anthropic, OpenAI, Gemini, Ollama) → Mocked with @patch
  • Email sending → Uses mail.mail records (real Odoo transactional mail, not sent)

Context Sharing

The ctx fixture allows steps to pass state:

@given('I enter topic "{topic}"')
def given_enter_topic(ctx, topic):
    ctx['topic'] = topic

@when('I click "Generate Now"')
def when_click_generate_now(odoo_env, ctx):
    topic = ctx.get('topic')  # Retrieve from context
    # ... use topic ...

Helper Functions

_make_mock_llm_response(topic) creates a realistic mock LLM response:

resp.text = f'<h1>{topic}</h1><p>Content...</p>'
resp.tokens_used = 850
resp.title = topic
resp.meta_title = f'{topic} — Enterprise Guide 2026'[:60]
resp.meta_description = f'A comprehensive guide to {topic}...'[:155]
resp.meta_keywords = 'AI, enterprise, trends'
resp.tags = ['Enterprise AI', 'AI Trends']
resp.social = MagicMock(
    twitter_a='...',
    twitter_b='...',
    bluesky_a='...',
    bluesky_b='...',
    mastodon='...',
    linkedin='...'
)

Scenario Coverage Map

Feature Scenarios Lines Coverage
blog_generation 3 42 happy path, draft, error
blog_scheduling 2 28 scheduled posts
llm_provider_selection 6 42 all 4 providers, errors, tokens
seo_population 1 11 SEO fields
notification_email 2 24 publish + draft email logic
TOTAL 14 147 100% of major workflows

Next Steps (Phase A: Test Environment)

  1. Set up template database in CI job
  2. Run BDD suite against live Odoo instance
  3. Verify email sending with Odoo's mail.mail system
  4. Verify image generation pipeline integration
  5. Add Playwright E2E for UI journey testing (1030 critical paths)
  6. Measure coverage (target: ≥80% on new code)

Troubleshooting

Issue Cause Fix
scenarios() not found pytest-bdd not installed pip install pytest-bdd
No scenarios found Feature file path incorrect Check scenarios('../features/...') paths
Step not defined Missing Given/When/Then definition Add step definition to test_bdd_steps.py
Mock not called Provider path wrong in patch Verify @patch('module.path.to.generate')
Blog.post not created LLMRouter not mocked Check _make_mock_llm_response() setup
Email not found Recipient email mismatch Verify ir.config_parameter recipient matches step

Conventions

  • Feature files: Separated by responsibility (one feature per file)
  • Scenario names: Complete sentences describing user intent ("... generates blog content")
  • Steps: Declarative, not imperative ("I enter" → "Given I enter", not "Click the field")
  • Given steps: Setup — configure API keys, create records
  • When steps: Execution — trigger wizard/router/email
  • Then steps: Assertion — verify state changes
  • Context: Share state between steps via ctx dict
  • Assertions: One per step method (exceptions for closely related checks)

References