OpenParable
Skip to content
← Back to skills

Bench Xref

Apps & Automation
bench-xref

Bench Xref is an Agent Skill for apps & automation published in the markuplint/markuplint repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Sync benchmark cross-reference blocks onto GitHub Issue bodies via `yarn bench:xref`, manage `tests/external/bench/issue-xref.config.ts` mappings, and run the pre-release checklist. Use when adding / removing / updating a primary or secondary mapping, when an issue closes upstream, when the xref CLI's output disagrees with what's on the issue body, or before cutting a release that touches a rule the benchmark covers. Trigger keywords: bench-xref, issue-xref, xref CLI, GitHub issue body, primary mapping, secondary mapping, umbrella block, pre-release checklist, bench-xref-audit workflow, marker version.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/markuplint/markuplint/blob/HEAD/.claude/skills/bench-xref/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/bench-xref, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

nu-validator Bench Cross-Reference Sync Skill

Configured GitHub issues carry a <!-- bench-xref:begin v1 --> ... <!-- bench-xref:end --> block at the end of their body. The block lists every fixture matching the issue's scope (filter regex) and the current verdict tally. Block content is generated from snapshots/diff/* and is idempotent — re-running only updates when the underlying numbers change.

Mappings live in tests/external/bench/issue-xref.config.ts.

Commands

Flags (--issue, --filter, --all, --write, --dry-run, --audit, --json) are parsed in tests/external/bench/xref-issue.ts — read it when unsure. Without --write, output goes to stdout and no network call is made.

Prerequisite: gh installed and authenticated.

Routine after a yarn bench:update:

yarn bench:xref --all --dry-run --write   # review intended diffs
yarn bench:xref --all --write             # apply

composeBody is idempotent and collapses multiple marker pairs into one block.

The block does not adjudicate who is right between markuplint and nu — that is a spec read per bench-triage.

Updating issue-xref.config.ts

EventAction
New Issue this bench can verifyAdd a primary mapping with a focused filter. Confirm with yarn bench:xref --issue <N>.
New Issue with no matching fixtureAdd a secondary mapping with a reason. Internal-bug Issues go here too.
Existing Issue closesRemove the entry. The umbrella block auto-derives from remaining primary mappings.
Issue scope shiftsAdjust filter. Rerun yarn bench:xref --issue <N>.
Body claim becomes inaccurate after bench rerunsAdd or edit a bodyOverride (factory reading an .md under tests/external/bench/issue-xref/).

Pre-release checklist

Before cutting a markuplint release that touches any rule the benchmark covers:

yarn bench:xref --audit     # prune CLOSED issues from the config
yarn bench:update           # regenerate coverage.json (Docker required)
yarn bench:xref --all --write

--audit is the cheapest gate; running it first prevents stale xref blocks from being pushed to issues that closed between releases.

CI: Bench xref audit workflow

.github/workflows/bench-xref-audit.yml runs the audit on relevant PRs and a weekly cron; it exits non-zero on any CLOSED entry, blocking PR merge until the mapping is removed.

Marker version

The v1 literal is hard-coded in the composeBody regex. To break format compat: bump BEGIN_MARKER / END_MARKER in xref-issue.ts, temporarily widen the regex to accept both versions, run yarn bench:xref --all --write once, then drop the old version from the regex.