OpenParable
Skip to content
← Back to skills

Cuvslam Onboard

Development
cuvslam-onboard

Cuvslam Onboard is an Agent Skill for development published in the nvidia-isaac/cuVSLAM repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Build, install, and run NVIDIA cuVSLAM and PyCuVSLAM from source or wheels. Covers environment setup, dataset preparation, and running examples for all tracking modes (stereo, mono, mono-depth, stereo-inertial, multi-camera) and SLAM (mapping, localization, loop closure). Use when asked to: build cuVSLAM, install PyCuVSLAM, set up cuVSLAM environment, run cuVSLAM examples, prepare KITTI/EuRoC/TUM datasets, run visual odometry, set up live camera tracking (RealSense/ZED/OAK-D/Orbbec), run cuVSLAM in Docker, or use cuVSLAM C++ tools.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/nvidia-isaac/cuVSLAM/blob/HEAD/cuvslam-skills/cuvslam-onboard/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/cuvslam-onboard, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

cuVSLAM Onboarding

Build, install, and run NVIDIA cuVSLAM — CUDA-accelerated visual odometry and SLAM.

For tracking/pose accuracy issues, see the cuvslam-troubleshoot skill instead.

For dataset-specific walkthroughs (EuRoC calibration, KITTI SLAM, TUM depth settings, multi-camera EDEX extraction), read references/dataset-guides.md.

For live camera setup (RealSense, ZED, OAK-D, Orbbec), read references/live-cameras.md.


Agent Interaction Guidelines

Before executing any setup step, ask the user for missing paths. Do not assume defaults.

  • Cloning the repo: Always ask — "Where would you like to clone the cuVSLAM repository? (e.g. ~/cuVSLAM)"
  • Dataset preparation: Always ask — "Where would you like to save the [KITTI/EuRoC/TUM/etc.] dataset? (e.g. ~/datasets/kitti)"
  • Virtual environment: If not already set up, ask — "Where would you like to create the Python virtual environment? (e.g. ~/cuVSLAM/.venv)"

Once the user provides a path, use it consistently throughout all subsequent commands in that session. Do not re-ask for the same path.


1. Requirements

  • Ubuntu 22+ (x86_64 or aarch64/Jetson)
  • CUDA Toolkit 12 or 13 (https://developer.nvidia.com/cuda/toolkit)
  • System packages: apt update && apt install g++ cmake git git-lfs python3-dev
  • CMake 3.19+, Python 3.9+

2. Install PyCuVSLAM (Quickest Path)

Pre-built wheels from https://github.com/nvidia-isaac/cuVSLAM/releases:

UbuntuPythonCUDAArch
22.043.1012, 13x86_64, aarch64
24.04+3.12+12, 13x86_64, aarch64
python3 -m venv .venv && source .venv/bin/activate
pip install cuvslam-<version>.whl
pip install -r examples/requirements.txt   # rerun-sdk, numpy, etc.

3. Build from Source

Clone

Ask the user: "Where would you like to clone the cuVSLAM repository?" before running these commands. Use their answer as <install-dir>.

git clone https://github.com/nvidia-isaac/cuVSLAM.git <install-dir>
cd <install-dir>

Build C++ library

cmake -S . -B build
cmake --build build --parallel $(nproc)

CMake options:

  • -DUSE_RERUN=ON — enable Rerun visualization for C++ tools
  • -DCUVSLAM_BUILD_SHARED_LIB=TRUE — build shared library (default)
  • -DUSE_CUDA=ON — use CUDA (default)

Install PyCuVSLAM from source

After building C++:

CUVSLAM_BUILD_DIR=$(pwd)/build pip install -e python/

Warning: Reinstall PyCuVSLAM after every C++ rebuild (scikit-build-core limitation).

Build on Jetson (remote ARM)

./copy_to_remote.sh <jetson-host>
ssh <jetson-host> 'CUVSLAM_SRC_DIR="<install-dir>"; CUVSLAM_DST_DIR="$CUVSLAM_SRC_DIR/build"; export CUVSLAM_SRC_DIR CUVSLAM_DST_DIR; "$CUVSLAM_SRC_DIR/build_release.sh"'
./copy_from_remote.sh <jetson-host>

Docker (with RealSense support)

# Ubuntu 22.04 + CUDA 12
docker build -f docker/Dockerfile.realsense-cu12 -t pycuvslam:realsense-cu12 .
./docker/run_docker.sh

# Ubuntu 24.04 + CUDA 13
docker build -f docker/Dockerfile.realsense-cu13 -t pycuvslam:realsense-cu13 .
./docker/run_docker.sh 24

Minimum drivers: CUDA 12 → driver ≥560, CUDA 13 → driver ≥580.

4. Tracking Modes

cuVSLAM supports these visual tracking modes:

ModeEnumUse case
StereoOdometryMode.Multicamera (0)Default. Two+ synchronized cameras
Stereo-InertialOdometryMode.Inertial (1)Stereo + IMU for robustness
Mono-Depth (RGB-D)OdometryMode.RGBD (2)Monocular + depth image
MonocularOdometryMode.Mono (3)Single camera (no scale)
MultisensorOdometryMode.Multisensor (4)Any-mix RGB / RGB-D cameras with optional IMU. Requires cuNLS-enabled build. Configure via MultisensorSettings. See examples/multisensor/.

5. Run Examples — Public Datasets

Environment setup (common to all examples)

source .venv/bin/activate    # if using venv
cd examples
pip install -r requirements.txt

KITTI (Stereo Odometry) — quickest demo

Ask the user: "Where would you like to save the KITTI dataset? (e.g. ~/datasets/kitti)" before downloading. Then create a symlink so the example script can find it: ln -s <dataset-dir> examples/kitti/dataset

cd examples/kitti
# Download: http://www.cvlibs.net/datasets/kitti/eval_odometry.php (grayscale, 22GB)
# Unzip so <dataset-dir>/sequences/00/image_0/*.png exists
# Symlink dataset into the example directory:
ln -s <dataset-dir> dataset
python3 track_kitti.py

SLAM with mapping + localization:

python3 track_kitti_slam.py     # maps, saves trajectory + map/data.mdb

EuRoC (Stereo-Inertial)

Ask the user: "Where would you like to save the EuRoC dataset? (e.g. ~/datasets/euroc)" before downloading. Then create a symlink: ln -s <dataset-dir> examples/euroc/dataset

cd examples/euroc
# Download MH_01_easy from https://doi.org/10.3929/ethz-b-000690084
# Extract mav0/ to <dataset-dir>/mav0/
ln -s <dataset-dir> dataset
cp sensor_cam0.yaml dataset/mav0/cam0/sensor_recalibrated.yaml
cp sensor_cam1.yaml dataset/mav0/cam1/sensor_recalibrated.yaml
cp sensor_imu0.yaml dataset/mav0/imu0/sensor_recalibrated.yaml
python3 track_euroc.py

TUM RGB-D (Mono-Depth)

Ask the user: "Where would you like to save the TUM RGB-D dataset? (e.g. ~/datasets/tum)" before downloading.

cd examples/tum
mkdir -p <dataset-dir>
wget https://cvg.cit.tum.de/rgbd/dataset/freiburg3/rgbd_dataset_freiburg3_long_office_household.tgz -O <dataset-dir>/fr3.tgz
tar -xzf <dataset-dir>/fr3.tgz -C <dataset-dir> && rm <dataset-dir>/fr3.tgz
ln -s <dataset-dir> dataset
cp freiburg3_rig.yaml dataset/rgbd_dataset_freiburg3_long_office_household/
python3 track_tum.py

Multi-Camera (Tartan Ground, 6 stereo pairs)

cd examples/multicamera_edex
pip install tartanair     # x86_64 only
python3 download_tartan.py
python3 track_multicamera_tartan.py

6. Run Examples — Live Cameras

See references/live-cameras.md for detailed setup per camera.

CameraStereoVIORGB-DMulti-cam
RealSenserun_stereo.pyrun_vio.pyrun_rgbd.pyrun_multicamera.py
ZEDrun_stereo.py—run_rgbd.py—
OAK-Drun_stereo.py———
Orbbecrun_stereo.py—run_rgbd.py—

7. C++ API

EuRoC C++ example

cmake -S . -B build
cmake --build build --target track_euroc
./build/bin/track_euroc /path/to/euroc/mav0

With Rerun: cmake -S . -B build -DUSE_RERUN=ON && cmake --build build --target track_euroc

C++ tools

ToolPurposeUsage
trackerCLI image-sequence tracking./bin/tracker config.cfg
cuvslam_api_launcherTrack, save map, localize./bin/cuvslam_api_launcher -dataset=<edex>
undistortRemove lens distortion./bin/undistort in.png calib.edex out.png
result_visualizerVisualize EDEX trajectoriespython3 tools/edex/result_visualizer/result_visualizer.py result.edex
bag2edexConvert ROS2 bag → EDEXpython3 bag_to_edex.py <bag> <out.edex>

8. SLAM Workflow

  1. Map: Run tracker with SLAM config to collect map
  2. Save: Map stored as map/data.mdb (LMDB)
  3. Localize: Load saved map, provide initial pose hint, call tracker.localize_in_map()
odom_cfg = cuvslam.Tracker.OdometryConfig(...)
slam_cfg = cuvslam.Tracker.SlamConfig(sync_mode=True)  # sync for reproducibility
tracker = cuvslam.Tracker(cuvslam.Rig(...), odom_cfg, slam_cfg)

odom_pose, slam_pose = tracker.track(...)

# Save map
tracker.save_map("map/")

# Later: localize
loc_settings = cuvslam.Tracker.SlamLocalizationSettings()
tracker.localize_in_map("map/", timestamp, pose_hint, loc_settings)

See examples/kitti/track_kitti_slam.py for the complete workflow.

9. Advanced Features

  • Static masks: Crop robot body / distorted edges via camera.border_top/bottom/left/right
  • Dynamic masks: Real-time segmentation masks as PyTorch GPU tensors — see examples/kitti/track_kitti_masks.py
  • Distortion models: pinhole, fisheye, brown, polynomial — see references/dataset-guides.md
  • Debug dump: Set config.debug_dump_directory to capture EDEX + images for offline analysis
  • Rerun visualization: All Python examples use Rerun; C++ needs -DUSE_RERUN=ON

10. ROS 2 Integration

Isaac ROS cuVSLAM wraps the C++ API as a ROS 2 node: