OpenParable
Skip to content
← Back to skills

Test Guide

Testing & Quality
test-guide

Test Guide is an Agent Skill for testing & quality published in the lablup/backend.ai repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Test-writing guide for Backend.AI — propose success/exception/edge scenarios first, refine them with the user, then implement while reporting per-scenario verification status. Covers fixtures, with_tables, mock repositories, pants test, optional TDD cadence.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/lablup/backend.ai/blob/HEAD/.claude/skills/test-guide/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/test-guide, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Test Guide

Write tests by agreeing on scenarios first, then verifying each one as you implement. The center of gravity is the scenario list and its verification status — not a fixed Red-Green-Refactor cadence (that is one valid way to write the tests; see §3).

1. Propose scenarios first

Before writing code or tests, draft the scenarios and get the user's input:

  1. Draft success / exception / edge scenarios from the requirement.
  2. Present them to the user for review — they add missing cases and correct expectations.
  3. Confirm the merged list. It is the contract for what "done" means.
## Test Target: {Feature}

### Success
1. {primary case} -> {expected}
2. {edge case that should succeed} -> {expected}

### Exception
1. {invalid input} -> raises {ErrorType}
2. {not found} -> raises {ErrorType}
3. {boundary} -> {empty/default behavior}

How to derive scenarios

Walk these axes — every one that applies to the feature becomes one or more scenarios:

AxisAskExample scenario
Happy pathWhat is the primary success?valid input -> entity created
VariationsOther valid inputs/states?optional field omitted -> default applied
Not foundA referenced entity is missing?unknown id -> raises XxxNotFound
ValidationInvalid/malformed input?negative quantity -> raises InvalidArgument
Conflict / stateDuplicate, or wrong current state?create existing name -> raises XxxConflict
PermissionCaller lacks rights?non-admin -> 403 / raises Forbidden
BoundaryEmpty, zero, max, end-of-page?offset past total -> empty list
Idempotency / concurrencyRepeat or parallel calls?re-run -> no duplicate side effect

Pull exception scenarios straight from the domain rules and the BackendAIError subclasses the code can raise (see manager/errors/AGENTS.md). Every scenario must name a concrete expected outcome or error type — "should fail" is not specific enough to test.

Propose first because the user catches missing cases far faster than reverse-engineering them from code, and the agreed list prevents scope drift.

2. Verify scenarios collaboratively

As you implement, report each scenario's status back to the user and keep it visible:

ScenarioTestStatus
returns record when presenttest_returns_record✅ pass
raises NotFound when missingtest_missing_raises✅ pass
pagination past end—⬜ not covered yet
  • Surface gaps (not covered) explicitly — never silently skip a scenario.
  • When the user adds a scenario mid-work, append it to the list and the table.
  • A scenario is done only when its test exists and passes.

3. Writing tests

One scenario = one test. Arrange (fixtures) → Act (call) → Assert.

class TestFeature:
    @pytest.fixture
    async def scenario_data(
        self, database_engine: ExtendedAsyncSAEngine
    ) -> AsyncGenerator[DataRow, None]:
        # setup with with_tables — see tests/AGENTS.md
        yield data

    async def test_returns_record_when_present(
        self, repository: FeatureRepository, scenario_data: DataRow
    ) -> None:
        result = await repository.method(scenario_data.id)
        assert result.field == expected
  • Fixtures: name by scenario (domain_with_record), return complete data, type as AsyncGenerator[T, None].
  • Repositories / models: real DB via with_tables; include FK deps, clean up child → parent. See tests/AGENTS.md.
  • Services / handlers: unit tests with mocked repositories — isolate business logic.
  • Types: annotate everything; test functions return -> None.
  • TDD (optional cadence): write the failing test first (confirm it fails for the right reason — ImportError/AttributeError), implement the minimum, then refactor while green.

4. Run and verify

Verify against the tests you added or changed — not a transitive sweep.

# the specific test(s) for the scenarios you implemented
pants test tests/manager/repositories/test_feature.py
pants test tests/.../test_feature.py::TestFeature::test_case

# quality, scoped to your change
pants fmt --changed-since=HEAD~1
pants fix --changed-since=HEAD~1
pants lint --changed-since=HEAD~1
pants check --changed-since=HEAD~1
pants test --changed-since=HEAD~1
  • Add a BUILD file with python_tests() to new test directories.
  • Fix all lint/type/test errors — never # noqa / # type: ignore / skip. See BUILDING.md.

Cross-References

  • tests/AGENTS.md — testing guidelines, with_tables, strategies
  • /repository-guide, /service-guide, /api-guide — layer patterns under test
  • BUILDING.md — quality enforcement