review-tests

Review Tests

Run the review-tests CLI and present the results to the user.

Steps

1. Run bactopia-review-tests via the wrapper script using the default text output (do NOT use --json):

   bash .claude/skills/review-tests/scripts/run-bactopia-review-tests.sh --bactopia-path /home/rpetit3/repos/bactopia/bactopia --silent
   

   If the user provided a timestamp argument (e.g., /review-tests 20260324_081306), add --run 20260324_081306.

2. Present the text output directly to the user. The CLI already produces a clean, well-formatted summary with tables. Do NOT parse JSON or write extra code to reformat -- just relay the output with your interpretation.

3. Add interpretation and context after showing the output:

   • For assertion_failure results, check the run parameters shown in the output:
     • If generate was true: snapshots were regenerated and the test was run a second time against them. These are real failures -- the workflow output does not match its own freshly-generated snapshot, meaning the output is non-deterministic or the test assertions are wrong. Flag these as needing investigation, NOT snapshot regeneration.
     • If generate was false (or not shown): snapshots may be stale. Note these likely need snapshot regeneration or investigation.
   • For undeclared_outputs results: these are files the tool produced in its work directory that are NOT declared in the module's results, logs, versions, or nf_logs output fields. Present each affected module with its undeclared file list (from the .outputs.txt log file). For each file, help the user decide:
     • Add to results: if the file is a real tool output users would want (e.g., a report, summary, or data file)
     • Add to logs: if the file is stderr/stdout from the tool itself
     • Add to .outputs-ignore: if the file is a staging artifact, intermediate, version-info side effect, or database file that should not be published The .outputs-ignore file lives at modules/{name}/tests/.outputs-ignore with one glob pattern per line (# comments, blank lines allowed). The staging/** directory is already ignored by default.
   • For suspiciously fast tests: note these likely exited early without running.
   • Summarize actionable items and suggested next steps.

4. If the text output is too large for a single response, summarize the key sections (overview, status breakdown, failures) and note that timing details are available on request. Use --json only as a fallback if the text output cannot be displayed.

Progressive Disclosure

The initial summary should be compact and scannable. When the user asks for deeper detail:

• Specific component: Read its stdout file at logs/{timestamp}/{tier}/{component}.stdout.txt using the Read tool
• Undeclared outputs: Read the component's .outputs.txt file at logs/{timestamp}/{tier}/{component}.outputs.txt for the full file list. Then read the module's main.nf to see the current results and logs fields and advise where each undeclared file should go.
• Abort errors: Read the nextflow.log for the component (focus on ERROR/WARN lines and last 50 lines). To find the log path, re-run with --json and check the nextflow_log field, or look in logs/{timestamp}/{tier}/{component}.stdout.txt for the path.
• Assertion details: Read the stdout file and look for specific assertion mismatch information

Do NOT read nextflow.log or stdout files during the initial summary.

Important Reminders

• CRITICAL: NEVER suggest "rerun with --update-snapshots" for non_reproducible failures -- that does NOT fix the root cause
• When params.generate is true, NEVER suggest snapshot regeneration for assertion failures -- snapshots were already regenerated during this run. These represent non-deterministic output or incorrect test assertions.
• Always read .stdout.txt files for diagnostics, NOT .stderr.txt
• The logs/{timestamp}/ directory contains tier subdirectories based on what was tested -- not all tiers are present in every run
• The .nf-test/ work directories under component test dirs only exist for failed tests (including undeclared_outputs failures -- preserved for review)
• If the user asks about a specific component, offer to read its stdout file in full and check for nextflow.log

Updating Baselines

Baselines file: conf/test-times.json

To update baselines after a clean all-pass run, add --update-baselines:

bash .claude/skills/review-tests/scripts/run-bactopia-review-tests.sh --bactopia-path /home/rpetit3/repos/bactopia/bactopia --silent --update-baselines

This writes actual runtimes from the current run into the baselines file and updates the _meta.updated timestamp. Only entries for tested components are updated; other tiers are left unchanged.

After updating, re-run without --update-baselines to confirm anomalies are resolved.

Interpreting Timing Anomalies

• generate=true vs generate=false: A generate=true run executes tests twice (generate snapshots, then test against them). If baselines were recorded from a generate=true run but the current run uses generate=false, tests will run at ~0.5x baseline. This is expected, not suspicious.
• Slow tests: May reflect newly added test cases rather than regressions. Check recent commits to the component's test file before flagging as a problem.
• Only flag anomalies as concerning when the generate parameter matches between the baseline run and the current run.

Self-Improvement

If you find yourself writing ad-hoc Python or bash to parse, explore, or extract data from the CLI output, that logic should be added to this skill or the underlying bactopia-review-tests CLI tool instead. Update the skill so future sessions don't need to reinvent it.

JSON Output (Fallback)

The --json flag is available as a fallback for programmatic access or when the text output is too large. Use it with --pretty for readable JSON. See bactopia-review-tests --help for details on JSON fields.