mo-bug-triage

Compatibility: designed for Codex CLI and compatible agents. Requires GitHub CLI (gh) with authenticated access (5000 req/hr) and repo write scope.

Enforcement Gates

---

TL;DR

Default lifecycle
  → New bug: kind/bug + needs-triage; no priority promise at entry
  → Analysis: verify repro, impact, owner, dependency, duplicates
  → Execute: promote only confirmed urgent/high-impact work to s-1/s0
  → Active non-emergency work: use s1 only when owned and committed
  → Downgrade: if not s0/s1 active work, move to deferred + comment
  → AI effort: do not run by default; assess only on explicit request or ai-* mode

Phase 1: LOCK STANDARDS   (~15min, 0 writes)
  → Sample 8-12 bugs by domain → lock lifecycle/severity/defer rules

Phase 2: BATCHED PROCESS  (~5min per batch of 50)
  → Each batch: fetch→classify action→report→snapshot→dry-run→update→progress
  → Batch N failure never touches Batch 1..N-1 (already committed)

Phase 3: FINAL VERIFY      (~10min)
  → Lifecycle audit + label count cross-check + defer-comment check; AI-label checks only for explicit AI modes

~310 issues ≈ 7 batches ≈ 30-45 min (limited by GitHub rate limit: 5000 req/hr).

---

Core Rules

Current GitHub Bug Workflow

Lifecycle Labels

Severity Hierarchy

Domain Downgrade Table

These domains default to deferred after analysis unless a higher-severity trigger is confirmed or an owner commits to near-term work:

Cold features list: stored procedures, CTE, window functions, collation, fulltext, backup/restore, role/DCL (GRANT/REVOKE), REPLACE INTO, trigger, event scheduler.

Note: LOAD DATA and import are NOT cold features — they are commonly-used ETL paths. Confirmed breakage can justify s0, but intake still starts at needs-triage.

s0 Investigation Triggers

These keywords are investigation triggers. They justify a closer look, not automatic promotion from needs-triage.

Classification Algorithm

1. Extract title (lowercase), body (lowercase) from issue JSON
2. If the issue is new/untriaged: ensure kind/bug + needs-triage; do not add active severity yet
3. Detect domains: is_partition, is_streaming, is_cold_feature, is_feature_request, duplicate/consolidation candidate
4. Detect evidence flags: panic/hung/OOM/data-integrity/resource-leak/common-path/release-blocker
5. Decide action:
   • promote-s-1: confirmed multi-tenant isolation/data exposure or explicit maintainer emergency
   • promote-s0: confirmed urgent/high-impact active bug with evidence and owner path
   • promote-s1: confirmed active non-emergency bug with owner/near-term commitment
   • defer: analyzed but not active s0/s1 work; requires comment
   • keep-needs-triage: insufficient information and no owner analysis yet
   • exclude: feature request or non-bug; remove from bug triage path per repo convention
6. Skip AI effort assessment by default. If and only if explicitly requested or running ai-assess/ai-final, estimate one AI effort label and record stage (issue-initial, pr-review, or user-assisted), confidence, and evidence in the report
7. Project: MOEngine-Compute default; MOEngine-Storage if logservice|tn|wal|replica|storage in title
8. If the user requested ai-final or PR AI-label review, re-evaluate AI label using actual code changes/tests/review complexity and update issue + PR labels

Key invariant: no issue leaves needs-triage for active severity without evidence and an owner/impact rationale. Keywords alone are never sufficient.

Deferred Comment Template

When moving to deferred, write a short, concrete comment:

Deferred after triage.

Reason: <unstable repro | missing dependency | non-critical path | consolidated into #NNNNN | feature request / not a bug | insufficient impact signal>
Current evidence: <one sentence>
To promote later: <specific signal needed, owner action, or linked issue/PR>

Skip List

Skip entirely — do not classify, do not update:

• heni02
• Ariznawlll

Severity Inflation Guard

• Before s-1: verify exactly matches multi-tenant isolation breach or explicit maintainer emergency. If serious but not s-1 and confirmed → s0; if unconfirmed → keep needs-triage.
• Before s0: verify actual panic/hung/OOM/integrity/common-function-break plus urgency/impact. If evidence is only title keywords → keep needs-triage.
• Before s1: verify there is an owner or near-term commitment. If not → deferred after analysis or keep needs-triage before analysis.
• Default for any post-analysis ambiguity → deferred with a clear reopen/promote condition.

AI Effort Assessment

Use AI labels to estimate how much human involvement is required, not severity or business priority. This is an opt-in assessment, not part of default needs-triage. Normalize spelling to lowercase: use ai-medium, not ai-Medium.

Assessment stages:

• issue-initial: rough label from issue title/body/logs/repro. Prefer ai-medium or ai-heavy when evidence is thin.
• pr-review: revise using linked PR diff, tests, review comments, files changed, and actual debugging burden.
• user-assisted: combine issue/PR evidence with user-provided context such as owner knowledge, hidden dependency, production signal, or known fix plan.

Assessment rules:

• Do not infer AI effort during plain intake or default kind/bug,needs-triage triage. Leave all ai-* labels untouched unless the user requested AI assessment.
• Apply at most one AI effort label at a time; remove the other four when updating.
• Record ai_label_stage, ai_confidence (low, medium, high), and one-sentence ai_rationale in reports.
• Escalate the label when correctness/security/consensus/storage/transaction risk is present, even if the diff looks small.
• De-escalate only with evidence: linked PR proved localized, deterministic tests pass, or user confirms a simple known fix.
• Near PR close, revise labels based on actual work rather than the initial estimate when an AI label exists or the user asks for ai-final.

Ownership Rule

• For issues/PRs assigned to you or linked to your work, update labels/comments proactively.
• Link PRs to issues, keep existing/requested issue and PR AI labels consistent near close, and close or defer issues explicitly.
• Do not wait for another person to drive routine status, downgrade, AI label correction, or closure hygiene.

---

Phase 1: LOCK STANDARDS (Read-Only, ~15 min)

Step 1.1: Discover Total Count

gh issue list -R matrixorigin/matrixone -l kind/bug -s open --limit 1 --json totalCount

Record total_count. Plan batches: ceil(total_count / 50).

Step 1.2: Query Existing Lifecycle Distribution

gh issue list -R matrixorigin/matrixone -l kind/bug -s open --limit 500 --json labels \
  --jq '.[].labels[].name' \
  | grep -E '^(needs-triage|deferred|severity/|ai-easy|ai-light|ai-medium|ai-heavy|ai-manual)
#x27; \
  | sort | uniq -c | sort -rn

Do not force a target ratio. Treat high active severity counts as an audit signal: severity/s-1 must remain rare, severity/s0/severity/s1 must have evidence/owners, and new work should accumulate in needs-triage until analyzed.

Step 1.3: Domain-Stratified Sampling

Fetch 8-12 issues covering every major domain (1-2 each):

gh issue list -R matrixorigin/matrixone -l kind/bug -s open --limit 100 \
  --json number,title,labels,assignees

Step 1.4: STANDARDS.md → User Approval Gate

Present sampled issues with proposed lifecycle actions. Produce file:

# MO Bug Triage Standards — [Date]

## Approved: false   ← MUST be set to true before Phase 2

## Lifecycle + Severity Rules (locked after approval)
... (Core Rules tables above)

## Sampled Examples (user-confirmed)
| # | Issue | Domain | Proposed Action | AI Estimate | Deferred Comment Required | User Decision |
|---|-------|--------|-----------------|-------------|---------------------------|---------------|
| 1 | ... | DML | promote-s0 | not requested | no | approved |
| 2 | ... | Partition | defer | not requested | yes: non-critical path, no owner | approved |

Wait for explicit user "approved"/"looks good", then:

sed -i 's/Approved: false/Approved: true/' STANDARDS.md

Step 1.5: Technical Gate (scripts MUST enforce)

All Phase 2 scripts must call before any GitHub write:

import sys, re
content = open("STANDARDS.md").read()
m = re.search(r'^## Approved:\s*(true|false)', content, re.MULTILINE)
if not m or m.group(1) != "true":
    print("FATAL: STANDARDS.md not approved. Run Phase 1 first.")
    sys.exit(1)

---

Phase 2: BATCHED PROCESS (~5 min/batch)

Each batch is a sealed unit: fetch → classify action → report → snapshot → dry-run → update → progress. Batch N failure never touches Batch 1..N-1.

Step 2a: Select Mode + Fetch One Batch

Choose the mode explicitly:

PAGE=$1
LABEL_QUERY="${LABEL_QUERY:-kind/bug,needs-triage}"
gh api -H "Accept: application/vnd.github+json" \
  "/repos/matrixorigin/matrixone/issues?state=open&labels=${LABEL_QUERY}&per_page=50&page=${PAGE}&sort=created&direction=desc" \
  --jq '.[] | {number,title,body,labels:[.labels[].name],assignees:[.assignees[].login],state,comments:.comments}' \
  > batch_${PAGE}_raw.json

Fallback: per_page=50 → 30 if rate-limited. Log actual page size.

Prefer REST API over GraphQL for reads — returns full data, no N+1 rate limit issues. GraphQL only for ProjectV2 writes.

Step 2b: Classify This Batch

Apply the Classification Algorithm from Core Rules. Output batch_N_classified.json with:

• action: keep-needs-triage, promote-s-1, promote-s0, promote-s1, defer, or exclude
• add_labels and remove_labels: exact lifecycle/severity label delta. Include AI label deltas only in explicit ai-assess/ai-final mode.
• rationale: one sentence explaining evidence and owner/impact reasoning
• deferred_comment: required when action == "defer"

Only when AI assessment is explicitly enabled, also output:

• ai_label: ai-easy, ai-light, ai-medium, ai-heavy, ai-manual, or unknown
• ai_label_stage: issue-initial, pr-review, user-assisted, or unknown
• ai_confidence: low, medium, or high
• ai_rationale: one sentence explaining expected human vs AI effort

Skip checking: filter out heni02 and Ariznawlll issues before classification.

Step 2c: Generate Batch Report

# MO Bug Triage — Batch N
**Generated**: 2026-07-01T10:23:45Z
**Source**: batch_N_classified.json (sha256: abc123...)
**DO NOT EDIT MANUALLY** — re-run classifier to regenerate.

## Batch Summary
| Metric | Value |
|--------|-------|
| Range | #25260 → #24997 |
| Total fetched | 50 |
| Skipped (heni02/Ariznawlll) | 3 |
| Classified actions | 47 |

## Action Distribution
| Action | Count |
|--------|-------|
| keep-needs-triage | 18 |
| promote-s-1 | 0 |
| promote-s0 | 4 |
| promote-s1 | 6 |
| defer | 19 |
| exclude | 0 |

## AI Effort Distribution (only when explicitly requested)
| Label | issue-initial | pr-review | user-assisted | unknown |
|-------|---------------|-----------|---------------|---------|
| ai-easy | 5 | 0 | 0 | 0 |
| ai-light | 7 | 0 | 0 | 0 |
| ai-medium | 16 | 0 | 0 | 0 |
| ai-heavy | 8 | 0 | 0 | 0 |
| ai-manual | 5 | 0 | 0 | 0 |
| unknown | 6 | 0 | 0 | 6 |

## Issue Details
| # | Title | Action | Labels Delta | AI Effort | Project | Deferred Comment | Rationale |
|---|-------|--------|--------------|-----------|---------|------------------|-----------|
| 25260 | ... | promote-s0 | +severity/s0 -needs-triage | not requested | MOEngine-Compute | no | confirmed INSERT panic on common path |
| 25261 | ... | defer | +deferred -needs-triage | not requested | MOEngine-Compute | yes | repro unstable; needs isolated testcase |

When AI assessment is explicitly enabled, replace AI Effort with ai-medium/issue-initial/medium style values and include ai_rationale.

Include generated_at timestamp + file checksum for drift detection.

Step 2d: Save Before-Snapshot (Rollback Safety)

Fetch current GitHub state for every issue in this batch → batch_N_before_snapshot.json:

{
  "generated_at": "2026-07-01T10:23:45Z",
  "issues": [
    {
      "number": 25260,
      "old_milestone": "MO-v1.1",
      "old_labels": ["kind/bug", "needs-triage"],
      "old_assignees": ["owner"],
      "old_comments_count": 3
    }
  ]
}

Step 2e: Dry-Run (5 issues)

Update first 5 issues. Verify labels and deferred comments on 2 of them:

gh issue view 25260 --json milestone,labels,comments \
  --jq '{milestone:.milestone.title,labels:[.labels[].name],last_comment:(.comments[-1].body // "")}'

Mismatch, missing deferred comment, or stale AI label in explicit AI mode → stop batch, investigate. Do not proceed.

Step 2f: Full Batch Update + Progress

After update completes, mandatory progress display:

========================================
BATCH 3 COMPLETE  ✅
Progress: ████████████████░░░░ 57%  (4/7 batches)
Running totals: needs-triage:118  deferred:42  s-1:1  s0:17  s1:28
Next: Batch 4 (#24510 → #24320)
========================================

Rollback Procedure (batch fails mid-update)

1. Read batch_N_before_snapshot.json
2. Restore each already-updated issue's old milestone and labels exactly
3. If the update log recorded newly-created deferred comment IDs, delete those comments; if deletion is not permitted, append a correction comment and stop
4. Never "fix forward" — always rollback and restart batch from clean state

# Restore old milestone
gh api -X PATCH /repos/matrixorigin/matrixone/issues/$NUMBER -f milestone="$OLD_MILESTONE"

# Restore labels: compute existing labels first, then remove labels not in snapshot and add labels from snapshot
gh issue edit $NUMBER --remove-label "$LABELS_TO_REMOVE" --add-label "$LABELS_TO_ADD"

# Delete a comment created by this failed batch when comment_id was logged
gh api -X DELETE /repos/matrixorigin/matrixone/issues/comments/$COMMENT_ID

---

Phase 3: FINAL VERIFY (Read-Only, ~10 min)

Step 3a: Lifecycle Audit

Automated checks:

• Alert if open kind/bug has none of needs-triage, deferred, severity/s-1, severity/s0, severity/s1
• Alert if severity/s0/severity/s1 issues lack owner/impact rationale in report
• Alert if deferred issues created in this run lack a comment matching the Deferred Comment Template
• Alert if partition/streaming/cold-feature issues remain active without confirmed trigger or owner commitment
• Alert if needs-triage items are old enough to indicate the candidate pool is not being drained
• Alert in explicit AI modes if an issue/PR has more than one AI effort label or an AI effort label without stage/confidence/rationale in the report

Step 3b: Label Count Cross-Validation

Assert: report counts match GitHub for needs-triage, deferred, severity/s-1, severity/s0, and severity/s1. In explicit AI modes, also assert counts for ai-easy, ai-light, ai-medium, ai-heavy, and ai-manual.

Mismatch → drift → regenerate + re-update affected batches.

Step 3c: Full Batch Spot-Check

Scan ALL 50 issues in one complete batch report. 100% of one batch catches systematic patterns that random 5-issue sampling misses: premature severity, missing comments, or owner gaps. In explicit AI modes, also check incorrect AI estimates.

Step 3d: Sampled API Verify

for num in $(shuf -e <all_numbers> | head -5); do
  gh issue view $num --json milestone,labels \
    --jq '{milestone:.milestone.title,labels:[.labels[].name]}'
done

Step 3e: AI Effort Final Check (explicit AI modes only)

Run this only when the user requests AI effort review or when existing AI labels are in scope. For PRs close to merge/close, compare issue and PR labels:

• If AI handled the fix directly and human review was only a scan, ensure ai-easy.
• If AI produced a useful draft but human tuned or made a bounded decision, ensure ai-light.
• If human and AI iterated with shared effort, ensure ai-medium.
• If AI only assisted and core logic stayed human-owned, ensure ai-heavy.
• If AI provided little practical help, ensure ai-manual.
• Remove the other four AI effort labels so only one remains.

Step 3f: Drift Detection

Compare generated_at in report headers vs file mtime. If file was edited after generation (mtime > generated_at + 5min) → the file was manually tampered → re-run classifier, re-update GitHub.

Step 3g: Final Summary

========================================
TRIAGE COMPLETE
========================================
Total open kind/bug: NNN
Actions applied:     NNN
Skipped:              XX (heni02/Ariznawlll)

Lifecycle Distribution (on GitHub):
  needs-triage: NNN
  deferred:      NN
  s-1:            N
  s0:            NN
  s1:            NN

AI Effort Distribution (only if explicit AI mode ran):
  ai-easy:       NN
  ai-light:      NN
  ai-medium:     NN
  ai-heavy:      NN
  ai-manual:     NN

All updated issues have explicit lifecycle state; deferred issues have comments; active severity issues have rationale. AI effort labels are unchanged unless explicit AI mode ran.
========================================

---

GitHub API Reference

Rate Limit Budget

300 issues × 2 calls = 600 requests. Realistic: 15-20 min with batch overhead.

Set Lifecycle Labels + Milestone

# Set milestone
gh issue edit $NUMBER --milestone "MO-202607"

# Intake normalization
gh issue edit $NUMBER --add-label "kind/bug,needs-triage"

# Promote to active severity: remove only labels that are actually present
gh issue edit $NUMBER --add-label "severity/s0" --remove-label "needs-triage,deferred,severity/s-1,severity/s1"

# Defer after analysis
gh issue edit $NUMBER --add-label "deferred" --remove-label "needs-triage,severity/s0,severity/s1,severity/s-1"
gh issue comment $NUMBER --body-file deferred_comment_${NUMBER}.md

# Revise AI effort label on issue or PR: add one, remove the other four
gh issue edit $NUMBER --add-label "ai-medium" --remove-label "ai-easy,ai-light,ai-heavy,ai-manual"
gh pr edit $PR_NUMBER --add-label "ai-medium" --remove-label "ai-easy,ai-light,ai-heavy,ai-manual"

gh issue edit with --add-label/--remove-label is simpler than REST PATCH. Prefer it.

Set Project (GraphQL — requires project OAuth scope)

Field IDs and Option IDs are project-specific. Query them once and cache.

mutation {
  updateProjectV2ItemFieldValue(input: {
    projectId: "PVT_..."
    itemId: "PVTI_..."
    fieldId: "PVTSSF_..."       # "Project" field
    value: { singleSelectOptionId: "..." }  # "MOEngine-Compute" or "MOEngine-Storage"
  }) { projectV2Item { id } }
}

If token lacks project scope: document assignment in report for manual allocation. Do not attempt and fail silently.

Fallback: curl (no gh CLI)

curl -s -H "Authorization: Bearer $GITHUB_TOKEN" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/repos/matrixorigin/matrixone/issues?state=open&labels=kind/bug&per_page=100&page=1"

---

Common Pitfalls (All from Real Sessions)

---

Script Checklist

Scripts accompanying this skill must:

• Check STANDARDS.md Approved: true gate before any GitHub write
• Normalize new kind/bug issues into needs-triage before assigning priority labels
• Require promotion rationale before adding severity/s-1, severity/s0, or severity/s1
• Require a deferred comment body before adding deferred
• Leave ai-* untouched during normal needs-triage; only in explicit AI modes, track exactly one of ai-easy, ai-light, ai-medium, ai-heavy, ai-manual with stage/confidence/rationale
• Accept --dry-run flag (default: first 5 issues per batch)
• Save batch_N_before_snapshot.json before updating each batch
• Print progress bar + running totals after each batch
• Detect generated_at vs file mtime drift (warn, don't block)
• Cross-validate lifecycle label counts in Phase 3; cross-validate AI label counts only for explicit AI modes
• Support --rollback batch_N to restore from snapshot
• Log created comment IDs so rollback can delete or correct failed-batch comments
• Log every API call to triage_YYYYMMDD.log

---

Refinement Loop (User Feedback Mid-Triage)

1. User says "change X" → pause current batch. DO NOT start new batch.
2. Run impact analysis: which processed batches affected? How many issues?
3. Show preview, get user confirmation:

⚠️  Rule change: "partition bugs without owner → deferred instead of s1"
    Affected batches: 1, 3, 5 (already processed)
    Issues to re-classify: 18
    Action changes: 13 (promote-s1→defer) + 2 (promote-s0 stays, confirmed panic) + 3 (no change)
    New comments required: 13
    Proceed? [y/N]

4. Re-classify affected batches only, regenerate reports (new timestamps)
5. Re-update GitHub for affected batches (snapshot → update labels/comments → verify)
6. Resume from next unprocessed batch