Agentic Testing
Testing & Qualityagentic-testingAgentic Testing is an Agent Skill for testing & quality published in the carlos-algms/agentic.nvim repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
MANDATORY before creating, editing, or reviewing tests in agentic.nvim, and before behavior changes that require TDD. Covers mini.test workflow, red/green rules, commands, mark-count checks, and which test references to load.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/carlos-algms/agentic.nvim/blob/HEAD/.agents/skills/agentic-testing/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/agentic-testing, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Agentic Testing
Framework
- mini.test with Busted-style emulation.
- Test files live next to source files as
<module>.test.lua. tests/holds the runner, helpers, functional tests, and integration tests.- The previous Busted/lazy.nvim setup is gone.
TDD
For every bug fix or behavioral change:
- Bootstrap missing symbols first so the test loads.
- Write the failing assertion.
- Run the focused test with
make test-file FILE=<path>and confirm it fails for behavior, not setup:- wrong: missing module, nil method, syntax error, unresolved import
- right: value/state/output mismatch
- Implement the minimum code to pass.
- Re-run
make test-file FILE=<path>. - Run the relevant full check.
- After adding or changing tests, reconcile the case count. This guards against
silently dropped OR unexpectedly generated tests. Both are failures.
- Compute EXPECTED cases by reading the file, not grepping:
- each static
it()= 1 case - each
it()inside afor/each/table-driven loop = the loop's iteration count (a singleit(line can emit many cases, or zero) - a grep of
it(is a lower bound, never the answer
- each static
- Read ACTUAL from
Total number of cases: Nin themake test-file FILE=<path>output. - EXPECTED MUST equal ACTUAL. Mismatch = a test was dropped, a loop is empty, or a generator misfired; stop and reconcile before claiming green. Eyeballing ACTUAL alone is NOT the check - you must derive EXPECTED first.
- Compute EXPECTED cases by reading the file, not grepping:
Pure refactors, formatting, and docs can skip red/green, but say that in the PR.
Commands
make test
make test-file FILE=lua/agentic/acp/agent_modes.test.lua
Use make test-file FILE=<path> for the red/green inner loop; it runs one file
in seconds. Never run make validate between iterations. After all .lua
edits, run make validate and fix until it passes - the task is not done until
it does. Pre-commit gate, not a per-test step.
For Lua or test changes, root AGENTS.md requires make validate after the
focused checks.
Before writing assertions
Read the exact helper APIs before using them:
tests/helpers/assert.luatests/helpers/spy.luatests/helpers/child.luawhen using child Neovim tests
Load references only when needed:
references/assert-spy.md: custom assert, spy, and stub API details.references/child-nvim.md: child process tests and RPC helpers.references/async-tests.md: scheduled/deferred code and mark-count traps.
Defaults
- Unit tests: co-located
<module>.test.lua. - Integration or functional tests:
tests/integration/ortests/functional/when behavior spans modules or needs real editor state. - ACP or transport-touching tests must stub
agentic.acp.acp_transportor any dependency that opens subprocesses or network calls. - Tests run sequentially in one Neovim process unless you explicitly use
tests.helpers.child. - Clean up buffers, windows, autocommands, globals, stubs, and spies.