OpenParable
Skip to content
← Back to skills

Isaac Sim Validator

Testing & Quality
isaac-sim-validator

Isaac Sim Validator is an Agent Skill for testing & quality published in the isaac-sim/IsaacSim repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Validate Isaac Sim simulation scripts and outputs before delivery. Rejects black frames, deprecated `omni.isaac.*` imports, hardcoded user paths/IPs, missing lights, wrong render mode, undersized SimulationApp configs, and bad RL output formats. Use when a sub-agent produces a script, a render, a training artifact, or any USD scene that's about to be handed to the user.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/isaac-sim/IsaacSim/blob/HEAD/skills/isaac-sim-validator/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/isaac-sim-validator, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Isaac Sim Validator

Final gate before delivery. Reject outputs that fail the checks below; do not soften results.

Universal checklist (apply to every output)

CheckPassOn fail
Script pathexists, readablereject: Script path invalid: <path>
File size> 1 KBreject: Script empty or corrupted
isaacsim importuses isaacsim.*, not omni.isaac.corereject: Use isaacsim.* namespace; omni.isaac.core is deprecated
SimulationApp configexplicit width and heightreject: SimulationApp without window args causes swapchain-capture size mismatch
Lights presentDomeLight (intensity >= 100) and DistantLight (>= 500)reject: No lighting; render will be black
Render modeRayTracedLighting for iteration; PathTracing only for hero shotsreject: PathTracing too slow for iteration; use RayTracedLighting
ACES tonemap/rtx/post/tonemap/op=4, filmIso 200 default / 600 deep-aisle / 400 aerialsuggest: Enable ACES (op=4) for contrast
Final render>= 150 KB and mean_RGB > 30reject: Render is black or low-energy; check lighting
User pathsno /home/<user>/, C:\Users\<name>\, or per-agent home pathsreject: Hardcoded user path; use $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR, or argparse
Network endpointsno bare IPs (192.168.x.x, 10.x.x.x) or non-localhost hostsreject: Hardcoded endpoint; parameterize via env var or config
Output dirconfigurable (env var, CLI, or function param)reject: Output dir hardcoded; accept as parameter or read $OUTPUT_DIR
VRAM headroomnvidia-smi headroom > 20% after settlewarn at >90% utilization; likely leak or scale issue
pkill -f "kit/kit" usedflag for kill+relaunch bugwarn: Verify zero kit processes before next launch
Sim duration>= 3 s of physicsreject: Sim ran < 3 s; physics has not settled

File structure validation (USD scenes)

ItemValidInvalidAction
$ISAAC_LAB_DIR/scripts/read-only referencewriting outputs thereredirect to $OUTPUT_DIR or $WORKSPACE_DIR/outputs/
get_assets_root_path() -> s3://...runtime OKbaked into delivered outputcache locally; reference via configurable path
Melting / Splat MDL-only shadersOK in raw formMDL-only at exportfilter out; MDL-only causes black render
MakeInstanceable for RLrequired (make_instanceable: true)absentreject
usdPreviewSurface on surfacesrequiredabsentreject; suggest adding UsdPreviewSurface
convex_decompose_meshfalse (with clean_sdf_dc)truerecommend false for stability
mpu=0.01OK for warehouse shell (13 m tiles)on a robotreject

Render validation (frame analysis)

MetricGoodBadAction
File size1–4 MB~82 KB82 KB = black, reject
Mean RGB80–160< 20reject: Too dark; add lights
Max RGB200–2500reject: No light in scene
Pixel variance> 15< 5flat = missing shadows; add accents
Color histogrammulti-modalsingle peakoverexposed or flat lighting; adjust intensity
Object visibility>= 3 robots or 50+ assets< 10 visiblesuggest: Scene too sparse; add set dressing
Shadow consistencyconsistent shadowsrandom black blobsreject: Shadow artifacts; check light cone bounds

RL training validation

IssueAction
AppLauncher not firstreject: AppLauncher must precede any isaaclab imports
Task __init__.py non-emptyreject: Task __init__.py must be empty; do not import configs
omni.isaac.lab importreject: Use isaaclab.*; omni.isaac.lab is deprecated
PhysX backend (when Newton applies)reject: Use Newton; it is the default on Kit 110
Missing PYTHONUNBUFFERED=1warn: Set PYTHONUNBUFFERED=1 to see logs in real time
No GPU memory loggingsuggest: Log torch.cuda.memory_allocated()/1e6 to W&B
Batch size exceeds VRAMsuggest: Reduce num_envs (try 2048)
Wrong checkpoint namereject: Must be model_XXXX.zip, not policy.pt
Single seedsuggest: Run seeds 42, 123, 456 for valid comparison

Asset & shell validation

InputCheckAction
docker run command-v mount for /workspace/outputreject: Output folder not mounted
Sub-agent spawnworkdir=$WORKSPACE_DIRreject: Sub-agent workdir hardcoded; use $WORKSPACE_DIR
Blender USD exportcorrect mpu, dual-shader (UsdPreviewSurface + MDL), correct export typereject if any missing
Scene complexityrenders within 10 settle stepsreject: Scene too complex; simplify assets
Ghost artifactscleanup of /dev/shm/carb-*reject: Use workspace_cleanup.py to clear /dev/shm/carb-*

Decision tree

graph TD
    A[Received Output] --> B{Has script?}
    B -->|No| C[Reject: "No script provided"]
    B -->|Yes| D{File size >1KB?}
    D -->|No| E[Reject: "Script empty or corrupted"]
    D -->|Yes| F{Python?}
    F -->|Yes| G{isaacsim.* imports?}
    G -->|No| H[Reject: "Use isaacsim.* not omni.isaac.core"]
    G -->|Yes| I{SimulationApp or --exec?}
    I -->|Neither| J[Reject: "Must use SimulationApp with window args OR isaac-sim.sh --exec"]
    I -->|SimulationApp| K{width and height set?}
    K -->|No| L[Reject: "SimulationApp requires explicit width and height"]
    K -->|Yes| M{Dome + Distant lights?}
    M -->|No| N[Reject: "Add DomeLight and DistantLight (min intensity 100 and 500)"]
    M -->|Yes| O{RayTracedLighting?}
    O -->|No| P[Reject: "Use RayTracedLighting for iteration"]
    O -->|Yes| Q{Render output exists?}
    Q -->|No| R[Reject: "No render output"]
    Q -->|Yes| S{>= 150 KB and mean_RGB > 30?}
    S -->|No| T[Reject: "Render is black, too dark, or corrupted"]
    S -->|Yes| U{Hardcoded user paths or bare IPs?}
    U -->|Yes| V[Reject: "Replace hardcoded user paths/IPs with env vars or args"]
    U -->|No| Y[VALID - deliver]

Training output

ItemAction
policy_only_XXXX.ptaccept
model_XXXX.zipaccept
log.csv or wandb URLaccept
results.json with episode dataaccept
No reward curve / metricsreject: Log episode/reward_mean and train/value_loss

Reply format

  • Pass: Valid: <one-line summary>
  • Fail: Invalid: <list of failed checks>
  • Suggest (only when otherwise valid): Could be improved: <tip>

Do not hedge ("I think", "maybe"). Do not deliver a black frame.

Input sources

  • isaac-sim-orchestrator, isaac-sim-rendering, other producer skills, and sub-agent outputs.
  • Invoked on: "validate this", "check this render", "review this code".

Feedback loop

  • Rejections feed back into the producing skill's failure-mode catalog.
  • Recurring failures become new validator rules.