OpenParable
Skip to content
← Back to skills

Skippy Cache Family Bench

Testing & Quality
skippy-cache-family-bench

Skippy Cache Family Bench is an Agent Skill for testing & quality published in the Mesh-LLM/mesh-llm repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use this skill when benchmarking Skippy exact-prefix cache across model families, comparing Skippy against llama-server, producing README benchmark tables, updating crates/skippy-cache/README.md evidence, or diagnosing cache benchmark gaps by family or Hugging Face use case.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/Mesh-LLM/mesh-llm/blob/HEAD/.agents/skills/skippy-cache-family-bench/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/skippy-cache-family-bench, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

skippy-cache-family-bench

Use this skill for reproducible Skippy cache benchmark evidence. The goal is to compare production cache payloads only: ResidentKv for dense families and KvRecurrent for recurrent/hybrid families. Do not report FullState as a production cache mode.

Workflow

  1. Run the reproducible wrapper. It builds by default, then runs full-GGUF baselines, Hugging Face use-case prompts, and the README report renderer:

    evals/skippy-cache-family-bench.sh /tmp/skippy-cache-family-bench
    
  2. For fast iteration after a build, skip the build step:

    SKIPPY_CACHE_SKIP_BUILD=1 evals/skippy-cache-family-bench.sh /tmp/skippy-cache-family-bench
    
  3. If running the pieces manually, keep these matched settings for full-GGUF family baselines:

    LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-cpu \
      python3 evals/skippy-cache-production-bench.py \
        --output-dir /tmp/skippy-cache-family-bench/full-gguf \
        --runtime-lane-count 1 \
        --llama-parallel 1 \
        --prefix-tokens 128
    
  4. Run the use-case benchmark matrix against the same family set:

    LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-cpu \
      python3 evals/skippy-cache-production-bench.py \
        --output-dir /tmp/skippy-cache-family-bench/use-cases \
        --runtime-lane-count 1 \
        --llama-parallel 1 \
        --prefix-tokens 128 \
        --use-case all
    
  5. Render README-ready tables from the combined JSON outputs:

    python3 evals/skippy-cache-family-report.py \
      --input /tmp/skippy-cache-family-bench/full-gguf/production-cache-bench.json \
      --input /tmp/skippy-cache-family-bench/use-cases/production-cache-bench.json \
      --output /tmp/skippy-cache-family-bench/readme-tables.md
    

Reporting Rules

  • Keep rows and columns ordered by related family: Qwen3Next, Falcon-H1, Llama, Qwen3 dense, DeepSeek2, GLM-4.7 Flash, GLM4, Gemma4 A4B, Gemma4 E4B, Gemma3, Gemma2, OLMo, MiniMax M2.7.
  • Always report Skippy versus llama-server for full-GGUF rows.
  • Keep DeepSeek3 in package-only evidence unless a machine can run a monolithic full-GGUF llama-server baseline.
  • Use one generated token and matched prefix tokens for apples-to-apples rows.
  • If a family fails correctness, leave the benchmark row out of promoted README evidence and explain the failure in crates/skippy-cache/TODO.md.
  • Preserve raw outputs under /tmp/... or another explicit run directory; do not paste ad hoc numbers without the backing production-cache-bench.json.