Validation Diff Gifs
Testing & Qualityvalidation-diff-gifsValidation Diff Gifs is an Agent Skill for testing & quality published in the isaac-sim/IsaacSim repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Generate pixel-difference GIF animations comparing benchmark validation captures against golden images. Use when the user asks to visualize validation failures, compare captured vs golden images, generate diff GIFs, or debug image tolerance issues.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/isaac-sim/IsaacSim/blob/HEAD/.claude/skills/validation-diff-gifs/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/validation-diff-gifs, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Validation Difference GIFs
Generate per-camera GIF animations showing the pixel-wise difference between captured benchmark images and their golden references. Useful for debugging validation tolerance failures.
Prerequisites
- ImageMagick (
composite,convert) must be available onPATH. - A completed validation capture run with matching directory structure to the golden data.
Paths
Captures and golden data live under the Isaac Sim build release directory:
standalone_examples/benchmarks/validation/captures/<run_name>/
standalone_examples/benchmarks/validation/golden_data/<benchmark_name>/
Both share a parallel directory tree (e.g. Robots/Robot_0/.../front_hawk/left/camera_left/rgb/).
Usage
Find the latest capture
ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1
Run the script
The bundled script is resolved via ${CLAUDE_SKILL_DIR}; falls back to the canonical repo path skills/validation-diff-gifs/scripts/:
bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" \
<captured_run_dir> \
<golden_benchmark_dir> \
[amplify] [fps]
| Argument | Default | Description |
|---|---|---|
captured_run_dir | (required) | Root of the capture run |
golden_benchmark_dir | (required) | Root of the golden data for the benchmark |
amplify | 10 | Multiply pixel differences by this factor for visibility |
fps | 5 | Frame rate for the output GIF |
Example
LATEST=$(ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1)
GOLDEN="standalone_examples/benchmarks/validation/golden_data/benchmark_robots_nova_carter_ros2"
bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" "$LATEST" "$GOLDEN"
Output
For each rgb/ directory under the capture, a diff_animation.gif is written alongside the captured PNGs:
<captured_run_dir>/Robots/.../front_hawk/left/camera_left/rgb/diff_animation.gif
- Black pixels = identical between captured and golden
- Brighter pixels = larger difference (amplified by the
amplifyfactor)
Interpreting results
- Uniform low-level noise across the frame → rendering non-determinism (likely acceptable)
- Bright regions concentrated on object edges → sub-pixel movement differences
- Entire frame bright → wrong timestamp match or completely different camera pose
- One camera consistently worse than others → possible per-camera issue (tick rate, initialization)