OpenParable
Skip to content
← Back to skills

Write Vibe Tests

Testing & Quality
write-vibe-tests

Write Vibe Tests is an Agent Skill for testing & quality published in the mistralai/mistral-vibe repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Write or refactor Mistral Vibe tests with proper decoupling. Use when adding behavior coverage, testing ports/adapters, replacing brittle mocks, creating fakes, adding characterization tests before refactors, or changing tests under tests/ for vibe/core, vibe/cli, vibe/acp, tools, config, sessions, skills, hooks, MCP, or setup.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/mistralai/mistral-vibe/blob/HEAD/.vibe/skills/write-vibe-tests/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/write-vibe-tests, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Write Vibe Tests

Use this skill to make Vibe architecture testable, not just well-shaped. Tests should protect observable behavior while allowing internals to move.

Core Principle

Test behavior through stable boundaries. Do not couple tests to private methods, internal call choreography, or temporary structure.

If a refactor changes no observable behavior but breaks many tests, the tests are probably coupled to implementation details.

Vibe Test Boundaries

Code areaPreferred test boundary
Core use cases and servicesPublic function/class API, typed events, model outputs, persisted state
ToolsBaseTool.invoke/run, typed args/results, permission behavior, user-facing errors
LLM/backend orchestrationAgentLoop events and fake backend outputs
ConfigPydantic model validation, layer merge outputs, migration results
SessionsSaved JSONL/metadata shape, resume/loader behavior, migration behavior
CLI widgetsTextual snapshots, posted messages, rendered user-visible state
ACPACP session updates and protocol-facing content, not core internals

Prefer Fakes Over Mocks

Prefer in-memory implementations and fake adapters that implement real contracts.

  • Put reusable test doubles in tests/stubs/ and name them Fake*.
  • Make fakes small and behavior-oriented.
  • Mock only at hard process, network, time, or third-party boundaries when a fake would be more complex than the behavior under test.
  • Avoid assertions like "method X was called with Y" unless the call itself is the observable contract.

Legacy Or Refactor Workflow

When code is hard to test:

  1. Find the smallest seam: function boundary, constructor dependency, protocol/port, wrapper, composition root, feature flag, or module boundary.
  2. Add characterization tests through the nearest public entry point.
  3. Capture current observable behavior, even if awkward.
  4. Refactor behind the seam in small steps.
  5. Replace broad characterization checks with clearer behavior/spec tests as the design improves.

Prefer "make it testable" refactors first: isolate I/O, extract pure functions, introduce ports/adapters where useful, or move construction out of business logic.

Test Shape

  • Use descriptive test names; do not add test docstrings.
  • Arrange, act, and assert clearly, but optimize for readability over ceremony.
  • Keep tests deterministic, fast, and explicit about failure.
  • Use autouse fixtures from tests/conftest.py for config/home/working-directory isolation.
  • Mark async tests with @pytest.mark.asyncio.
  • Mock outbound HTTP with respx.
  • Use the narrowest relevant test first, then broaden when shared contracts are touched.

Avoid

  • Testing private methods as the primary coverage for behavior.
  • Asserting intermediate internal state when user-visible output, emitted events, files, or return values can be asserted.
  • Building mocks that mirror the implementation.
  • Adding abstractions only to satisfy a test.
  • Writing tests that require a specific internal file split or call order when the domain behavior is unchanged.

Verification

After Python test/code changes, run:

uv run ruff format .
uv run ruff check --fix .

Then run the targeted tests:

uv run pytest <test-path-or-node-id>

Run uv run pyright when signatures, models, protocols, or shared contracts changed.