OpenParable
Skip to content
← Back to skills

Skippy Spec Bench

Testing & Quality
skippy-spec-bench

Skippy Spec Bench is an Agent Skill for testing & quality published in the Mesh-LLM/mesh-llm repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use this skill when testing or benchmarking target/draft GGUF pairs for speculative decoding compatibility, tokenizer agreement, draft acceptance rate, or staged verification behavior.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/Mesh-LLM/mesh-llm/blob/HEAD/.agents/skills/skippy-spec-bench/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/skippy-spec-bench, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

skippy-spec-bench

Use this skill for target/draft speculative compatibility work.

What It Checks

  • Target and draft tokenization agreement.
  • Baseline target decode versus draft-verified decode.
  • Draft acceptance/rejection behavior.
  • Batched verification and checkpoint/restore behavior.
  • Recurrent-state implications for rollback.

Repo Notes

The old source repo used a standalone llama-spec-bench crate. It may not be present in this mesh checkout yet, so verify available packages before running commands:

cargo metadata --no-deps --format-version 1 | jq -r '.packages[].name' | sort

If the spec bench is imported, keep it as a diagnostics/benchmark tool. Do not make normal mesh serving depend on it.