i4h-workflow-dataset-teleop

i4h Workflow — Teleop Record

Purpose

Record episodes for an agentic env via teleoperation (keyboard, SO-ARM leader, or VR) into HDF5. Use when the user wants to teleop or record human demos.

Base Code

These steps drive the i4h-workflows base code (the workflows/agentic/ tree). To reuse an existing checkout, set I4H_WORKFLOWS to its path (no clone happens). Otherwise this resolves the current repo, or clones to ~/i4h-workflows — pick that default without prompting. Run every command below from the resolved root:

# Resolve the i4h-workflows base code (provides workflows/agentic/).
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/agentic" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/i4h-workflows}"
  [ -d "$ROOT/workflows/agentic" ] || git clone https://github.com/isaac-for-healthcare/i4h-workflows "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"; cd "$ROOT"

Basics

• Env config (source of truth): workflows/agentic/config/environments/<env>.yaml — robot.type, zenoh.camera_names, and the task for <env>.
• Teleop runs through arena/run.sh --teleop.
• Device support is env-specific. Check arena/run.sh --env <env> --help for valid --teleop-device values.

Controls

Reserved keys (consistent across devices):

Device-specific keybindings (move, rotate, gripper, mode switches) are printed by the teleop process at startup and vary by --teleop-device. Report them to the user from the log; they cannot drive the sim without them. See "Surface Device Keybindings".

Stop from terminal:

workflows/agentic/stop.sh arena --env <env>

Known Devices

For other envs, consult arena/run.sh --env <env> --help.

Run

REPO_ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"; [ -d "$REPO_ROOT/workflows/agentic" ] || REPO_ROOT="$HOME/i4h-workflows"
ENV_ID=scissor_pick_and_place
RUNS_ROOT="${REPO_ROOT}/workflows/agentic/runs"
RUN_DIR="${RUNS_ROOT}/teleop_${ENV_ID}_$(date +%Y%m%d_%H%M%S)"
mkdir -p "${RUN_DIR}/data" "${RUN_DIR}/logs"
ln -sfn "${RUN_DIR}" "${RUNS_ROOT}/.latest"

"${REPO_ROOT}/workflows/agentic/arena/run.sh" \
  --env "${ENV_ID}" \
  --teleop \
  --teleop-device <device> \
  --episodes 3 \
  --record-to "${RUN_DIR}/data/demo.hdf5" \
  2>&1 | tee "${RUN_DIR}/logs/teleop.log"

Surface Device Keybindings

The teleop process prints its keybinding table to stdout shortly after launch (look for sections such as Keybindings, Controls, Key Map, mode-switch lines, or any block enumerating keys → actions). Background the run, wait for the table to appear, extract it from the log, and report it to the user before they need to drive the sim.

# After launching teleop in the background and tailing the log:
sed -n '/Keybind\|Controls\|Key Map\|Current Mode\|Mode\] Switched/,/^$/p' "${RUN_DIR}/logs/teleop.log"

If the block is multi-section (e.g. BOTH_HANDS, BASE_NAV, LEFT_HAND, RIGHT_HAND modes), include every mode in the report. Append the reserved keys above so the user has one consolidated reference.

Notes

• --record-to must be absolute. The recorder resolves relative paths against workflows/agentic/arena (its CWD) and writes to a nested orphan dir. ${RUN_DIR}/data/demo.hdf5 built from ${REPO_ROOT} is absolute.
• Use --save-all-episodes only when failed attempts must be kept.

Verify

• ${RUN_DIR}/data/demo.hdf5 exists.
• Log contains run complete: N/M episodes succeeded.

Prerequisites

• Workflow set up via [[i4h-workflow-setup]] (the .venv must exist).
• A valid env id and a --teleop-device it supports (see Known Devices or arena/run.sh --env <env> --help).
• The chosen teleop device available (keyboard, SO-ARM leader, or VR).
• An absolute --record-to HDF5 path (relative paths resolve against arena's CWD).

Limitations

• Device support is env-specific; not every device works with every env.
• --record-to must be absolute or the recording lands in a nested orphan dir.
• By default only successful episodes are saved; use --save-all-episodes to keep failed attempts.
• Device-specific keybindings are only printed at startup; they cannot be known before launching.

Troubleshooting

• Error: .venv not found / teleop fails to launch - Cause: workflow not set up. Fix: run [[i4h-workflow-setup]] first.
• Error: invalid --teleop-device for env - Cause: device not supported by that env. Fix: pick a value from arena/run.sh --env <env> --help (or Known Devices).
• Error: HDF5 written to an unexpected/nested location - Cause: --record-to was relative. Fix: pass an absolute path built from ${REPO_ROOT}.
• Error: keybindings missing / cannot drive the sim - Cause: the device keybinding table was not surfaced. Fix: background the run, extract the table from the log (see Surface Device Keybindings).

Final Response

Report env, device, the device-specific keybinding table extracted from the log, requested vs saved episodes, HDF5 path, log path.