Skip to main content

ADR-022: Testcontainers for Integration Tests

Status​

Implemented

Date​

2025-01-16 (Retrospective)

Decision Makers​

  • QA Team - Testing strategy
  • Backend Team - Test infrastructure

Layer​

Testing

  • ADR-001: PostgreSQL with pgvector
  • ADR-011: Redis Single Instance Strategy

Supersedes​

  • SQLite for testing (not compatible with pgvector)

Depends On​

  • ADR-001: PostgreSQL with pgvector

Context​

Integration tests need real database behavior:

  1. pgvector Compatibility: SQLite doesn't support pgvector
  2. Production Parity: Tests should match production behavior
  3. Isolation: Each test run needs clean state
  4. Performance: Tests should run quickly
  5. CI/CD: Must work in GitHub Actions

Key constraints:

  • pgvector extension requires PostgreSQL
  • Tests must be isolated from each other
  • Need fast teardown/setup
  • Must work locally and in CI
  • Can't use shared test database

Decision​

We adopt Testcontainers for integration tests with real PostgreSQL:

Key Design Decisions​

  1. Testcontainers Python: Docker container management
  2. PostgreSQL + pgvector: Same as production
  3. Per-Session Containers: Reuse across test module
  4. Automatic Cleanup: Containers removed after tests
  5. CI Support: GitHub Actions with Docker-in-Docker

Test Setup​

import pytest
from testcontainers.postgres import PostgresContainer

@pytest.fixture(scope="session")
def postgres_container():
    """Spin up PostgreSQL with pgvector for test session."""
    with PostgresContainer(
        image="pgvector/pgvector:pg15",
        user="test",
        password="test",
        dbname="test_db",
    ) as postgres:
        yield postgres

@pytest.fixture
def test_db(postgres_container):
    """Get database session for individual test."""
    engine = create_engine(postgres_container.get_connection_url())
    Base.metadata.create_all(engine)

    with Session(engine) as session:
        yield session
        session.rollback()  # Cleanup

Redis Container​

@pytest.fixture(scope="session")
def redis_container():
    """Spin up Redis for test session."""
    with RedisContainer("redis:7-alpine") as redis:
        yield redis

CI Configuration​

# .github/workflows/test.yml
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      # Docker-in-Docker for Testcontainers
      docker:
        image: docker:24-dind
    steps:
      - uses: actions/checkout@v4
      - run: pytest tests/integration/

Consequences​

Positive​

  • Production Parity: Same database as production
  • pgvector Support: Full vector functionality
  • Isolation: Each test run is independent
  • No Mocking: Real database behavior
  • Automatic Cleanup: No orphaned containers

Negative​

  • Startup Time: Container spin-up adds latency (~3-5s)
  • Docker Required: Must have Docker available
  • Resource Usage: Containers use memory/CPU
  • CI Complexity: Docker-in-Docker setup

Neutral​

  • Image Caching: First run slow, subsequent faster
  • Port Management: Automatic random port assignment

Alternatives Considered​

1. SQLite​

  • Approach: In-memory SQLite for speed
  • Rejected: No pgvector support, different SQL dialect

2. Shared Test Database​

  • Approach: Single PostgreSQL for all tests
  • Rejected: Test isolation issues, cleanup complexity

3. Mock Database​

  • Approach: Mock all database calls
  • Rejected: Doesn't test actual queries, maintenance burden

Implementation Status​

  • Core implementation complete
  • Tests written and passing
  • Documentation updated
  • Migration/upgrade path defined
  • Monitoring/observability in place

Implementation Details​

  • Fixtures: backend/tests/conftest.py
  • Integration Tests: backend/tests/integration/
  • CI Config: .github/workflows/test.yml
  • Docs: docs/development/testing/testing-strategy-for-fixes.md

Compliance/Validation​

  • Automated checks: Tests run in CI on every PR
  • Manual review: Test coverage reviewed weekly
  • Metrics: Test duration, pass/fail rates

LLM Council Review​

Review Date: 2025-01-16 Confidence Level: High (100%) Verdict: APPROVED WITH CONDITIONS

Quality Metrics​

  • Consensus Strength Score (CSS): 0.95
  • Deliberation Depth Index (DDI): 0.88

Council Feedback Summary​

Testcontainers is the correct choice since pgvector requirement renders lightweight alternatives insufficient. Per-session scope mitigates latency concerns but introduces isolation risks.

Key Concerns Identified:​

  1. Test Isolation Risk: Per-session reuse creates "dirty state" where Test A impacts Test B
  2. Docker-in-Docker Unnecessary: GitHub Actions already has Docker daemon; DinD adds complexity and flakiness
  3. Image Versioning: Relying on :pg15 tag risks upstream changes breaking tests

Required Modifications:​

  1. Transaction Rollback for Isolation:
    • scope="session" for Docker container (spin up once)
    • scope="function" for DB connection/transaction (clean state per test)
    • Wrap each test in transaction with ROLLBACK during teardown
  2. Socket Mapping for CI:
    • Remove docker:dind service; use host Docker socket
    • Set TESTCONTAINERS_RYUK_DISABLED=true (ephemeral CI runners don't need cleanup)
  3. Pin Image Digests: Use specific SHA digest, not just :pg15
  4. Explicit Health Checks: Wait for "database system is ready" log, not just port open
  5. Resource Limits: Set CPU/Memory limits to prevent OOM during parallel tests

Modifications Applied​

  1. Documented transaction rollback isolation pattern
  2. Added socket mapping for GitHub Actions
  3. Documented Ryuk disable for CI
  4. Added image digest pinning recommendation

Council Ranking​

  • All models reached strong consensus
  • gemini-3-pro: Best Response (isolation strategy)
  • gpt-5.2: Strong (CI configuration)

References​

ADR-022 | Testing Layer | Implemented