quality-gate-pipeline

Quality Gate Pipeline

Overview

Single quality gate that every agent passes through before output is accepted. Replaces three formerly separate skills (accuracy-validation, verification-before-completion, task-review-correction) with a unified pipeline of three phases that run in sequence.

Constraints

• Do not deliver output containing unverified claims; pause and verify first
• Do not claim completion without fresh verification evidence from this session
• Do not reference test results or tool output from earlier in the conversation — re-run and show current output
• Do not substitute reasoning or explanation for actual evidence
• Max 3 review-correction cycles before escalating to the Orchestrator
• Each correction cycle must reduce total defect count; flat or increasing defects trigger escalation

The Three Phases

Phase 1: Self-Validation (before delivery)

Every agent runs this checklist mentally before presenting output.

Factual Accuracy

• All file paths referenced actually exist (verify with tool, don't assume)
• All function/class/variable names match what's in the codebase
• Version numbers, API signatures, and config values are verified, not recalled from training
• No statistics or citations are fabricated
• Claims about an external tool or product's UI (menu paths, button/control labels, screen layouts, settings locations) are backed by a citation the agent actually has — a fetched doc, a user-provided screenshot/description, or tool output — not recalled from training as if current and authoritative. When no citation exists, say so plainly (e.g. "I don't have a verified/current view of this menu — check the tool's current UI") rather than describing specifics; this is the same Low confidence fallback as any other recalled/guessed claim (see Confidence Assessment below).

Instruction Fidelity

• Output addresses what the user actually asked, not a reinterpretation
• All acceptance criteria from the task are met
• No scope creep beyond the request
• Constraints from the agent persona are respected

Internal Consistency

• No contradictions within the output
• Code samples compile/run conceptually (correct syntax, valid imports)
• Referenced earlier decisions are accurately recalled (if unsure, re-read from memory/)

Confidence Assessment

Hallucination Detection Signals

Strong signals (likely hallucination):

• Referencing a file, function, or API that was never read in this session
• Quoting specific numbers without a source
• Describing behavior that contradicts tool observations
• Generating imports for packages not in dependencies
• Describing a specific control, menu path, or screen in an external tool/product without a citation the agent can point to

When a signal fires: Pause → Verify (use tools) → Correct → Log (hallucination_detected: true in metrics)

Phase 2: Verification Evidence (before completion claims)

Iron Law: No completion claims without fresh verification evidence. Skipping any step is falsification, not verification.

The Gate Function:

1. IDENTIFY — What command proves your claim? (e.g., npm test, cargo build)
2. RUN — Execute the command fresh and completely. Not from cache, not from memory.
3. READ — Read the complete output and exit code. Don't skim.
4. VERIFY — Does the output actually confirm your claim? If not, report actual status.
5. ONLY THEN — Make the claim, with supporting evidence pasted.

Required Evidence (all tasks):

1. Tests pass: Run the whole test suite. Paste output with pass/fail counts.
2. Build succeeds: If the project has a build step, run it. Paste output.
3. Lint clean: If the project has a linter, run it. Paste output.
4. No regressions: Test count should not decrease.
5. Coverage & Mutation (necessary-not-sufficient): Coverage thresholds and mutation score gate entry but do not alone gate exit. Saturating at 100% line coverage and 1.0 mutation score on small slices is expected and does not indicate structural quality. They are required to pass but are not the final signal.
6. Structural Quality gate: Before claiming "done", the work must carry a clean-or-triaged structure-review + complexity-review signal. Options:
   • Clean: Both agents produced no error/warning findings on the latest diff.
   • Triaged: Every error/warning finding is logged as a deferred item (with owner + tracking reference) per the Phase 3 exit criteria — and surfaced in the completion summary.
   • Waiver: For hotfixes, documentation-only changes, or work pre-approved by the Orchestrator as out-of-scope for structural review, include a waiver statement: structural-review-waiver: <reason> in the completion summary. Waivers are surfaced, not hidden.

Quality Ownership: green means the entire suite, not just the tests your change touched. A failing test is a failing test regardless of whether this change caused it — "already broken" / "not my diff" is not a pass. A red signal you observe must be fixed, or explicitly surfaced and triaged (file an issue, or record a quarantine with a reason via /triage) and reported as not green. Never claim completion over a red suite by attributing the failure to someone else's change. You own the quality state you can see, not just your delta.

Additional Evidence by task type:

Evidence Format:

## Verification
- Tests: `npm test` → 47 passed, 0 failed (output below)
- Build: `npm run build` → success, 0 warnings
- Lint: `npm run lint` → 0 errors, 0 warnings

Red Flag Language — stop and verify when you catch yourself saying:

• "should work now" / "should be fixed" / "probably" / "I believe"
• Expressing satisfaction before running verification
• Preparing commits without verification output

Phase 3: Review-Correction Loop (post-delivery rework)

Activated when output is returned for rework, during peer review, or when self-reviewing before delivery (complements Phase 1).

Defect Severity:

Correction Scope:

• Isolated fix: Self-contained; fix doesn't affect other outputs
• Cascading fix: Implies other outputs may also be wrong; verify related work
• Rework: Fundamental approach flawed; redo from requirements

Review Checklist:

1. Requirements compliance — all acceptance criteria addressed, no silent scope changes
2. Correctness — intended result, edge cases handled, no regressions
3. Completeness — all files touched, no TODOs remain, integration points addressed
4. Consistency — matches project conventions, no contradictions
5. Quality — appropriately simple, readable, sufficient documentation

Iteration Rules:

• Max 3 review-correction cycles before escalation
• Each cycle must reduce total defect count
• If defects increase or stay flat after 2 cycles, escalate to Orchestrator
• Exit criteria: all critical and major defects resolved (fixed and re-verified). Minor defects may be deferred — but deferred ≠ resolved: each must be logged with an owner and a tracking reference and surfaced in the completion summary, never folded into "done." A slice carrying deferred items is "complete with known follow-ups," not silently "complete."

Escalation: Summarize defect pattern and attempted corrections → escalate to Orchestrator for re-routing → log with escalation_reason in task metrics.

When to Apply Each Phase

Output

Phase 1: Confidence-scored validation — report failures only. Phase 2: Verification evidence block with tool output. Phase 3: Defect table (severity, scope, status: fixed/deferred/escalated).

Integration

• Performance Metrics: hallucination_detected, rework_cycles, defects_found are logged automatically by hooks/task_completion_metrics.py on task completion — no skill-level logging step needed. To surface non-default values, populate .claude/session-metrics.json with the relevant fields before the session ends.
• Context Summarization: When context is high, increase Phase 1 rigor
• Human Oversight Protocol: Escalation from Phase 3 feeds into the approval gate system