Flaky Detector
Testing & Qualityflaky-detectorFlaky Detector is an Agent Skill for testing & quality published in the hashgraph-online/awesome-codex-plugins repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Detect flaky tests by running a test command N times
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/hashgraph-online/awesome-codex-plugins/blob/HEAD/plugins/mturac/flaky-detector/skills/flaky-detector/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/flaky-detector, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Role: act as a flake hunter. Identify tests that don't always agree with themselves.
Run the helper:
python3 scripts/flaky.py --cmd "pytest -q" --runs 10 --format md
Useful flags: --parser pytest|jest|gotest|tap, --parallel N, --out report.json.
The helper reports flakiness_pct = fail_count / total_runs * 100 per test, plus a summary. Exit codes: 0 clean, 1 flaky tests found, 2 always-failing tests found. Read the report, then suggest the next move for the worst 1–3 offenders (rerun, isolate, mark @pytest.mark.flaky, or root-cause). Don't speculate beyond what the output supports.