Cuvslam Onboard
Developmentcuvslam-onboardCuvslam Onboard is an Agent Skill for development published in the nvidia-isaac/cuVSLAM repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Build, install, and run NVIDIA cuVSLAM and PyCuVSLAM from source or wheels. Covers environment setup, dataset preparation, and running examples for all tracking modes (stereo, mono, mono-depth, stereo-inertial, multi-camera) and SLAM (mapping, localization, loop closure). Use when asked to: build cuVSLAM, install PyCuVSLAM, set up cuVSLAM environment, run cuVSLAM examples, prepare KITTI/EuRoC/TUM datasets, run visual odometry, set up live camera tracking (RealSense/ZED/OAK-D/Orbbec), run cuVSLAM in Docker, or use cuVSLAM C++ tools.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/nvidia-isaac/cuVSLAM/blob/HEAD/cuvslam-skills/cuvslam-onboard/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/cuvslam-onboard, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
cuVSLAM Onboarding
Build, install, and run NVIDIA cuVSLAM — CUDA-accelerated visual odometry and SLAM.
- Repo: https://github.com/nvidia-isaac/cuVSLAM
- Python API docs: https://nvidia-isaac.github.io/cuVSLAM/python/
- C++ API docs: https://nvidia-isaac.github.io/cuVSLAM/cpp/
- Technical report: https://arxiv.org/abs/2506.04359
For tracking/pose accuracy issues, see the cuvslam-troubleshoot skill instead.
For dataset-specific walkthroughs (EuRoC calibration, KITTI SLAM, TUM depth settings, multi-camera EDEX extraction), read references/dataset-guides.md.
For live camera setup (RealSense, ZED, OAK-D, Orbbec), read references/live-cameras.md.
Agent Interaction Guidelines
Before executing any setup step, ask the user for missing paths. Do not assume defaults.
- Cloning the repo: Always ask — "Where would you like to clone the cuVSLAM repository? (e.g.
~/cuVSLAM)" - Dataset preparation: Always ask — "Where would you like to save the [KITTI/EuRoC/TUM/etc.] dataset? (e.g.
~/datasets/kitti)" - Virtual environment: If not already set up, ask — "Where would you like to create the Python virtual environment? (e.g.
~/cuVSLAM/.venv)"
Once the user provides a path, use it consistently throughout all subsequent commands in that session. Do not re-ask for the same path.
1. Requirements
- Ubuntu 22+ (x86_64 or aarch64/Jetson)
- CUDA Toolkit 12 or 13 (https://developer.nvidia.com/cuda/toolkit)
- System packages:
apt update && apt install g++ cmake git git-lfs python3-dev - CMake 3.19+, Python 3.9+
2. Install PyCuVSLAM (Quickest Path)
Pre-built wheels from https://github.com/nvidia-isaac/cuVSLAM/releases:
| Ubuntu | Python | CUDA | Arch |
|---|---|---|---|
| 22.04 | 3.10 | 12, 13 | x86_64, aarch64 |
| 24.04+ | 3.12+ | 12, 13 | x86_64, aarch64 |
python3 -m venv .venv && source .venv/bin/activate
pip install cuvslam-<version>.whl
pip install -r examples/requirements.txt # rerun-sdk, numpy, etc.
3. Build from Source
Clone
Ask the user: "Where would you like to clone the cuVSLAM repository?" before running these commands. Use their answer as
<install-dir>.
git clone https://github.com/nvidia-isaac/cuVSLAM.git <install-dir>
cd <install-dir>
Build C++ library
cmake -S . -B build
cmake --build build --parallel $(nproc)
CMake options:
-DUSE_RERUN=ON— enable Rerun visualization for C++ tools-DCUVSLAM_BUILD_SHARED_LIB=TRUE— build shared library (default)-DUSE_CUDA=ON— use CUDA (default)
Install PyCuVSLAM from source
After building C++:
CUVSLAM_BUILD_DIR=$(pwd)/build pip install -e python/
Warning: Reinstall PyCuVSLAM after every C++ rebuild (scikit-build-core limitation).
Build on Jetson (remote ARM)
./copy_to_remote.sh <jetson-host>
ssh <jetson-host> 'CUVSLAM_SRC_DIR="<install-dir>"; CUVSLAM_DST_DIR="$CUVSLAM_SRC_DIR/build"; export CUVSLAM_SRC_DIR CUVSLAM_DST_DIR; "$CUVSLAM_SRC_DIR/build_release.sh"'
./copy_from_remote.sh <jetson-host>
Docker (with RealSense support)
# Ubuntu 22.04 + CUDA 12
docker build -f docker/Dockerfile.realsense-cu12 -t pycuvslam:realsense-cu12 .
./docker/run_docker.sh
# Ubuntu 24.04 + CUDA 13
docker build -f docker/Dockerfile.realsense-cu13 -t pycuvslam:realsense-cu13 .
./docker/run_docker.sh 24
Minimum drivers: CUDA 12 → driver ≥560, CUDA 13 → driver ≥580.
4. Tracking Modes
cuVSLAM supports these visual tracking modes:
| Mode | Enum | Use case |
|---|---|---|
| Stereo | OdometryMode.Multicamera (0) | Default. Two+ synchronized cameras |
| Stereo-Inertial | OdometryMode.Inertial (1) | Stereo + IMU for robustness |
| Mono-Depth (RGB-D) | OdometryMode.RGBD (2) | Monocular + depth image |
| Monocular | OdometryMode.Mono (3) | Single camera (no scale) |
| Multisensor | OdometryMode.Multisensor (4) | Any-mix RGB / RGB-D cameras with optional IMU. Requires cuNLS-enabled build. Configure via MultisensorSettings. See examples/multisensor/. |
5. Run Examples — Public Datasets
Environment setup (common to all examples)
source .venv/bin/activate # if using venv
cd examples
pip install -r requirements.txt
KITTI (Stereo Odometry) — quickest demo
Ask the user: "Where would you like to save the KITTI dataset? (e.g.
~/datasets/kitti)" before downloading. Then create a symlink so the example script can find it:ln -s <dataset-dir> examples/kitti/dataset
cd examples/kitti
# Download: http://www.cvlibs.net/datasets/kitti/eval_odometry.php (grayscale, 22GB)
# Unzip so <dataset-dir>/sequences/00/image_0/*.png exists
# Symlink dataset into the example directory:
ln -s <dataset-dir> dataset
python3 track_kitti.py
SLAM with mapping + localization:
python3 track_kitti_slam.py # maps, saves trajectory + map/data.mdb
EuRoC (Stereo-Inertial)
Ask the user: "Where would you like to save the EuRoC dataset? (e.g.
~/datasets/euroc)" before downloading. Then create a symlink:ln -s <dataset-dir> examples/euroc/dataset
cd examples/euroc
# Download MH_01_easy from https://doi.org/10.3929/ethz-b-000690084
# Extract mav0/ to <dataset-dir>/mav0/
ln -s <dataset-dir> dataset
cp sensor_cam0.yaml dataset/mav0/cam0/sensor_recalibrated.yaml
cp sensor_cam1.yaml dataset/mav0/cam1/sensor_recalibrated.yaml
cp sensor_imu0.yaml dataset/mav0/imu0/sensor_recalibrated.yaml
python3 track_euroc.py
TUM RGB-D (Mono-Depth)
Ask the user: "Where would you like to save the TUM RGB-D dataset? (e.g.
~/datasets/tum)" before downloading.
cd examples/tum
mkdir -p <dataset-dir>
wget https://cvg.cit.tum.de/rgbd/dataset/freiburg3/rgbd_dataset_freiburg3_long_office_household.tgz -O <dataset-dir>/fr3.tgz
tar -xzf <dataset-dir>/fr3.tgz -C <dataset-dir> && rm <dataset-dir>/fr3.tgz
ln -s <dataset-dir> dataset
cp freiburg3_rig.yaml dataset/rgbd_dataset_freiburg3_long_office_household/
python3 track_tum.py
Multi-Camera (Tartan Ground, 6 stereo pairs)
cd examples/multicamera_edex
pip install tartanair # x86_64 only
python3 download_tartan.py
python3 track_multicamera_tartan.py
6. Run Examples — Live Cameras
See references/live-cameras.md for detailed setup per camera.
| Camera | Stereo | VIO | RGB-D | Multi-cam |
|---|---|---|---|---|
| RealSense | run_stereo.py | run_vio.py | run_rgbd.py | run_multicamera.py |
| ZED | run_stereo.py | — | run_rgbd.py | — |
| OAK-D | run_stereo.py | — | — | — |
| Orbbec | run_stereo.py | — | run_rgbd.py | — |
7. C++ API
EuRoC C++ example
cmake -S . -B build
cmake --build build --target track_euroc
./build/bin/track_euroc /path/to/euroc/mav0
With Rerun: cmake -S . -B build -DUSE_RERUN=ON && cmake --build build --target track_euroc
C++ tools
| Tool | Purpose | Usage |
|---|---|---|
tracker | CLI image-sequence tracking | ./bin/tracker config.cfg |
cuvslam_api_launcher | Track, save map, localize | ./bin/cuvslam_api_launcher -dataset=<edex> |
undistort | Remove lens distortion | ./bin/undistort in.png calib.edex out.png |
result_visualizer | Visualize EDEX trajectories | python3 tools/edex/result_visualizer/result_visualizer.py result.edex |
bag2edex | Convert ROS2 bag → EDEX | python3 bag_to_edex.py <bag> <out.edex> |
8. SLAM Workflow
- Map: Run tracker with SLAM config to collect map
- Save: Map stored as
map/data.mdb(LMDB) - Localize: Load saved map, provide initial pose hint, call
tracker.localize_in_map()
odom_cfg = cuvslam.Tracker.OdometryConfig(...)
slam_cfg = cuvslam.Tracker.SlamConfig(sync_mode=True) # sync for reproducibility
tracker = cuvslam.Tracker(cuvslam.Rig(...), odom_cfg, slam_cfg)
odom_pose, slam_pose = tracker.track(...)
# Save map
tracker.save_map("map/")
# Later: localize
loc_settings = cuvslam.Tracker.SlamLocalizationSettings()
tracker.localize_in_map("map/", timestamp, pose_hint, loc_settings)
See examples/kitti/track_kitti_slam.py for the complete workflow.
9. Advanced Features
- Static masks: Crop robot body / distorted edges via
camera.border_top/bottom/left/right - Dynamic masks: Real-time segmentation masks as PyTorch GPU tensors — see
examples/kitti/track_kitti_masks.py - Distortion models: pinhole, fisheye, brown, polynomial — see
references/dataset-guides.md - Debug dump: Set
config.debug_dump_directoryto capture EDEX + images for offline analysis - Rerun visualization: All Python examples use Rerun; C++ needs
-DUSE_RERUN=ON
10. ROS 2 Integration
Isaac ROS cuVSLAM wraps the C++ API as a ROS 2 node:
- GitHub: https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam
- Docs: https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/index.html
- Supports Humble on Jetson (JetPack 6.1/6.2) and x86_64 (Ubuntu 22.04+)