OpenParable
Skip to content
← Back to skills

Validation Diff Gifs

Testing & Quality
validation-diff-gifs

Validation Diff Gifs is an Agent Skill for testing & quality published in the isaac-sim/IsaacSim repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Generate pixel-difference GIF animations comparing benchmark validation captures against golden images. Use when the user asks to visualize validation failures, compare captured vs golden images, generate diff GIFs, or debug image tolerance issues.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/isaac-sim/IsaacSim/blob/HEAD/.claude/skills/validation-diff-gifs/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/validation-diff-gifs, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Validation Difference GIFs

Generate per-camera GIF animations showing the pixel-wise difference between captured benchmark images and their golden references. Useful for debugging validation tolerance failures.

Prerequisites

  • ImageMagick (composite, convert) must be available on PATH.
  • A completed validation capture run with matching directory structure to the golden data.

Paths

Captures and golden data live under the Isaac Sim build release directory:

standalone_examples/benchmarks/validation/captures/<run_name>/
standalone_examples/benchmarks/validation/golden_data/<benchmark_name>/

Both share a parallel directory tree (e.g. Robots/Robot_0/.../front_hawk/left/camera_left/rgb/).

Usage

Find the latest capture

ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1

Run the script

The bundled script is resolved via ${CLAUDE_SKILL_DIR}; falls back to the canonical repo path skills/validation-diff-gifs/scripts/:

bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" \
    <captured_run_dir> \
    <golden_benchmark_dir> \
    [amplify] [fps]
ArgumentDefaultDescription
captured_run_dir(required)Root of the capture run
golden_benchmark_dir(required)Root of the golden data for the benchmark
amplify10Multiply pixel differences by this factor for visibility
fps5Frame rate for the output GIF

Example

LATEST=$(ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1)
GOLDEN="standalone_examples/benchmarks/validation/golden_data/benchmark_robots_nova_carter_ros2"

bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" "$LATEST" "$GOLDEN"

Output

For each rgb/ directory under the capture, a diff_animation.gif is written alongside the captured PNGs:

<captured_run_dir>/Robots/.../front_hawk/left/camera_left/rgb/diff_animation.gif
  • Black pixels = identical between captured and golden
  • Brighter pixels = larger difference (amplified by the amplify factor)

Interpreting results

  • Uniform low-level noise across the frame → rendering non-determinism (likely acceptable)
  • Bright regions concentrated on object edges → sub-pixel movement differences
  • Entire frame bright → wrong timestamp match or completely different camera pose
  • One camera consistently worse than others → possible per-camera issue (tick rate, initialization)