isaac-sim-validator
Pre-delivery QA gate for standalone Isaac Sim libraries and Kit scripts. Level 3 executes untrusted Python unsandboxed and needs --confirm-execute.
How do I install this agent skill?
npx skills add https://github.com/isaac-sim/isaacsim --skill isaac-sim-validatorIs this agent skill safe to install?
- Gen Agent Trust Hubwarn
This skill provides a validation framework for Isaac Sim scripts. It includes a high-privilege validation mode (Level 3) that executes target Python scripts unsandboxed on the host system to perform runtime checks. While the skill includes clear documentation, warnings, and requires an explicit confirmation flag to prevent accidental execution, the capability to run arbitrary local code presents a significant security surface.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
Isaac Sim Validator
WARNING: Level 3 executes the target script with your full user privileges. There is no container or sandbox isolation. Only use --confirm-execute on scripts you fully trust.
Purpose
Reject common delivery defects at the owning boundary: standalone library or Kit application/extension.
Levels 1–2 are static checks only. Level 3 runs the target Python script under timeout with the calling user's privileges. Level 3 requires an explicit --confirm-execute (or CONFIRM_EXECUTE=1) opt-in; it is not a sandbox.
Prerequisites
- Standalone libraries need their configured build or installed test environment.
- Kit checks need a built Isaac Sim tree; a GPU is required only for GPU-backed checks.
Limitations
- Levels 1–2 cannot prove runtime correctness for every workload.
- Level 3 executes untrusted script content only after
--confirm-execute; there is no container/firejail isolation. - Thresholds target common demo/SDG failures, not all product features.
Troubleshooting
| Error / symptom | Cause | Solution |
|---|---|---|
| Kit extension not found | Wrong $ISAAC_SIM_DIR or stale build | Point env vars at _build/linux-x86_64/release or rebuild |
| Black or empty frames | Missing lights or non-RTX render mode | Add dome/key light; confirm RTX / PathTracing settings |
| Hang on stage load or first render | MDL compile or oversized stage | Follow isolation steps in isaac-sim-troubleshooting |
Final gate before delivery. Reject outputs that fail the checks below; do not soften results.
For a standalone library, validate a normal-process public import, declared
package/native closure, relocation, SDK consumer when applicable, and
isaacsim.common.logging.Logger diagnostics. Do not require SimulationApp,
rendering, or a GPU.
Kit-only validation helper
| Script | Purpose | Arguments |
|---|---|---|
scripts/validate_sim.sh | Validate sim | positional args per script header (see script) |
Running scripts
From agent runtimes that expose skill execution helpers, invoke helpers with run_script():
run_script("scripts/validate_sim.sh", args=[])
From a built Isaac Sim tree, run the same file with ./python.sh (Linux) or python.bat (Windows) from _build/*/release, or execute shell helpers directly when they do not require the simulator.
Kit simulation checklist
| Check | Pass | On fail |
|---|---|---|
| Script path | exists, readable | reject: Script path invalid: <path> |
| File size | > 1 KB | reject: Script empty or corrupted |
isaacsim import | uses isaacsim.*, not omni.isaac.core | reject: Use isaacsim.* namespace; omni.isaac.core is deprecated |
SimulationApp config | explicit width and height | reject: SimulationApp without window args causes swapchain-capture size mismatch |
| Lights present | DomeLight (intensity >= 100) and DistantLight (>= 500) | reject: No lighting; render will be black |
| Render mode | RayTracedLighting for iteration; PathTracing only for hero shots | reject: PathTracing too slow for iteration; use RayTracedLighting |
| ACES tonemap | carb rtx.post.tonemap.op = 4; filmIso 200 default / 600 deep-aisle / 400 aerial | suggest: Enable ACES (op=4) for contrast |
| Final render | >= 150 KB and mean_RGB > 30 | reject: Render is black or low-energy; check lighting |
| User paths | no /home/<user>/, hardcoded C:\path\to\..., or per-agent home paths; prefer %USERPROFILE%, env vars, or argparse | reject: Hardcoded user path; use $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR, or argparse |
| Network endpoints | no bare IPs (192.168.x.x, 10.x.x.x) or non-localhost hosts | reject: Hardcoded endpoint; parameterize via env var or config |
| Output dir | configurable (env var, CLI, or function param) | reject: Output dir hardcoded; accept as parameter or read $OUTPUT_DIR |
| VRAM headroom | nvidia-smi headroom > 20% after settle | warn at >90% utilization; likely leak or scale issue |
pkill -f "kit/kit" used | flag for kill+relaunch bug | warn: Verify zero kit processes before next launch |
| Sim duration | >= 3 s of physics | reject: Sim ran < 3 s; physics has not settled |
File structure validation (USD scenes)
| Item | Valid | Invalid | Action |
|---|---|---|---|
$ISAAC_LAB_DIR/scripts/ | read-only reference | writing outputs there | redirect to $OUTPUT_DIR or $WORKSPACE_DIR/outputs/ |
get_assets_root_path() -> s3://... | runtime OK | baked into delivered output | cache locally; reference via configurable path |
Melting / Splat MDL-only shaders | OK in raw form | MDL-only at export | filter out; MDL-only causes black render |
MakeInstanceable for RL | required (make_instanceable: true) | absent | reject |
usdPreviewSurface on surfaces | required | absent | reject; suggest adding UsdPreviewSurface |
convex_decompose_mesh | false (with clean_sdf_dc) | true | recommend false for stability |
mpu=0.01 | OK for warehouse shell (13 m tiles) | on a robot | reject |
Render validation (frame analysis)
| Metric | Good | Bad | Action |
|---|---|---|---|
| File size | 1–4 MB | ~82 KB | 82 KB = black, reject |
| Mean RGB | 80–160 | < 20 | reject: Too dark; add lights |
| Max RGB | 200–250 | 0 | reject: No light in scene |
| Pixel variance | > 15 | < 5 | flat = missing shadows; add accents |
| Color histogram | multi-modal | single peak | overexposed or flat lighting; adjust intensity |
| Object visibility | >= 3 robots or 50+ assets | < 10 visible | suggest: Scene too sparse; add set dressing |
| Shadow consistency | consistent shadows | random black blobs | reject: Shadow artifacts; check light cone bounds |
Video validation
For any delivered MP4/GIF, extract and inspect at least start/middle/end frames. Reject or recut the presentation copy if the first frames show render warm-up, the subject is occluded, or the camera loses the task. Keep the complete source render and metrics when a shorter presentation cut is made.
RL training validation
| Issue | Action |
|---|---|
AppLauncher not first | reject: AppLauncher must precede any isaaclab imports |
Task __init__.py non-empty | reject: Task __init__.py must be empty; do not import configs |
omni.isaac.lab import | reject: Use isaaclab.*; omni.isaac.lab is deprecated |
| PhysX backend (when Newton applies) | reject: Use Newton; it is the default on Kit 110 |
Missing PYTHONUNBUFFERED=1 | warn: Set PYTHONUNBUFFERED=1 to see logs in real time |
| No GPU memory logging | suggest: Log torch.cuda.memory_allocated()/1e6 to W&B |
| Batch size exceeds VRAM | suggest: Reduce num_envs (try 2048) |
| Wrong checkpoint name | reject: Must be model_XXXX.zip, not policy.pt |
| Single seed | suggest: Run seeds 42, 123, 456 for valid comparison |
Asset & shell validation
| Input | Check | Action |
|---|---|---|
docker run command | -v mount for /workspace/output | reject: Output folder not mounted |
| Sub-agent spawn | workdir=$WORKSPACE_DIR | reject: Sub-agent workdir hardcoded; use $WORKSPACE_DIR |
| Blender USD export | correct mpu, dual-shader (UsdPreviewSurface + MDL), correct export type | reject if any missing |
| Scene complexity | renders within 10 settle steps | reject: Scene too complex; simplify assets |
| Ghost artifacts | cleanup of /dev/shm/carb-* | reject: Use workspace_cleanup.py to clear /dev/shm/carb-* |
Kit decision tree
graph TD
A[Received Output] --> B{Has script?}
B -->|No| C[Reject: "No script provided"]
B -->|Yes| D{File size >1KB?}
D -->|No| E[Reject: "Script empty or corrupted"]
D -->|Yes| F{Python?}
F -->|Yes| G{isaacsim.* imports?}
G -->|No| H[Reject: "Use isaacsim.* not omni.isaac.core"]
G -->|Yes| I{SimulationApp or --exec?}
I -->|Neither| J[Reject: "Must use SimulationApp with window args OR isaac-sim.sh --exec"]
I -->|SimulationApp| K{width and height set?}
K -->|No| L[Reject: "SimulationApp requires explicit width and height"]
K -->|Yes| M{Dome + Distant lights?}
M -->|No| N[Reject: "Add DomeLight and DistantLight (min intensity 100 and 500)"]
M -->|Yes| O{RayTracedLighting?}
O -->|No| P[Reject: "Use RayTracedLighting for iteration"]
O -->|Yes| Q{Render output exists?}
Q -->|No| R[Reject: "No render output"]
Q -->|Yes| S{>= 150 KB and mean_RGB > 30?}
S -->|No| T[Reject: "Render is black, too dark, or corrupted"]
S -->|Yes| U{Hardcoded user paths or bare IPs?}
U -->|Yes| V[Reject: "Replace hardcoded user paths/IPs with env vars or args"]
U -->|No| Y[VALID - deliver]
Training output
| Item | Action |
|---|---|
policy_only_XXXX.pt | accept |
model_XXXX.zip | accept |
log.csv or wandb URL | accept |
results.json with episode data | accept |
| No reward curve / metrics | reject: Log episode/reward_mean and train/value_loss |
Reply format
- Pass:
Valid: <one-line summary> - Fail:
Invalid: <list of failed checks> - Suggest (only when otherwise valid):
Could be improved: <tip>
Do not hedge ("I think", "maybe"). Do not deliver a black frame.
Input sources
isaac-sim-orchestrator,isaac-sim-rendering, other producer skills, and sub-agent outputs.- Invoked on Isaac Sim deliverables: "validate this Isaac Sim script", "check this Isaac Sim render", "review this sim output".
Automated validation
Run the checks above from the CLI with scripts/validate_sim.sh:
# Levels 1–2: static only
bash scripts/validate_sim.sh --level <1|2> <script.py> [--output-dir <dir>] [--timeout <seconds>]
# Level 3: EXECUTES the script — requires explicit opt-in
bash scripts/validate_sim.sh --level 3 --confirm-execute <script.py> [--output-dir <dir>] [--timeout <seconds>]
# or: CONFIRM_EXECUTE=1 bash scripts/validate_sim.sh --level 3 <script.py> ...
Feedback loop
- Rejections feed back into the producing skill's failure-mode catalog.
- Recurring failures become new validator rules.
- Standing rubric:
references/quality-criteria.md. Record each iteration inreferences/feedback-log.md.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/isaac-sim/isaacsim/isaac-sim-validator">View isaac-sim-validator on skillZs</a>