Review Topic Local Vs Remote Execution
Testing & Qualityreview-topic-local-vs-remote-executionReview Topic Local Vs Remote Execution is an Agent Skill for testing & quality published in the roboflow/inference repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Load when a PR touches how a step/model runs in-process vs. over the network: Workflow-block execution-mode dispatch (self._step_execution_mode, StepExecutionMode.LOCAL/REMOTE, run_locally/run_remotely), a block get_restrictions()/get_air_gapped_availability(), backend/runtime routing env (USE_INFERENCE_MODELS, DISABLED_INFERENCE_MODELS_BACKENDS, WORKFLOWS_REMOTE_API_TARGET, inference_models_adapters.py), CUDA/Jetson/ONNX gating, or a new block with network/LAN/same-process assumptions.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/roboflow/inference/blob/HEAD/.claude/skills/review-topic-local-vs-remote-execution/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/review-topic-local-vs-remote-execution, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Review topic: Local vs remote / hosted execution compatibility
When this applies
Load when a diff does any of:
- Adds/edits a
run()dispatch onself._step_execution_mode(LOCAL vs REMOTE) or arun_locally/run_remotelypair. - Adds/edits a block's
get_restrictions()orget_air_gapped_availability()where the block reaches the network or needs same-process / LAN access (PTZ/ONVIF camera, local filesystem, GPU-resident handles). - Touches runtime routing env:
USE_INFERENCE_MODELS,DISABLED_INFERENCE_MODELS_BACKENDS,WORKFLOWS_STEP_EXECUTION_MODE,WORKFLOWS_REMOTE_API_TARGET,LOCAL_INFERENCE_API_URL,_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT, or backend selection ininference/core/models/inference_models_adapters.py/inference/models/utils.py. - Gates behavior on hardware/deps: CUDA version, Jetson (
jetson,l4t, JP5/JP6), ONNX execution providers, TRT — where one runtime path is fixed/tested but siblings are not. - Changes hosted/serverless/dedicated cold-start / processing-time headers or WebRTC/modal worker plumbing.
This is a behavior/content trigger, not a single directory — the paths above are hints.
Cross-refs (owned elsewhere, not reviewed here):
- Serverless auth/usage fail-open behavior →
review-topic-auth-and-tenant-security. - Stateful-block restriction contract (in-process trackers/counters/cooldown/append-log needing a SOFT restriction) →
review-topic-workflow-state-management.
Review checklist
- BLOCK —
run()dispatches onStepExecutionModewith an explicitelse: raise ValueErrorfor unknown modes (missing else swallows new modes silently). - BLOCK — a runtime-routing change (
USE_INFERENCE_MODELS,DISABLED_INFERENCE_MODELS_BACKENDS, platform default) leaves BOTH the on and off value working and tested; the legacy ONNX registry entries are still present. Breaking the DEFAULT runtime with no working fallback is a production regression. - BLOCK — a block that raises / hard-fails on a runtime the platform will still offer carries a matching
RuntimeRestriction(rightseverity,applies_to_step_execution_modes,applies_to_runtimes); metadata that lies makes the UI offer a path that fails at runtime. - FLAG —
run_locallyandrun_remotelyhave matching signatures and forward every manifest param; a param wired into only one path is a divergence users hit only on the other runtime. - FLAG — a new network/LAN/same-process block overrides
get_air_gapped_availability()if it needs internet, and adds a HARD restriction excluding hosted/dedicated (LAN) or REMOTE step execution (same-process). - FLAG — a platform-specific default (
_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT, Windows/Jetson forks) keeps the non-default OS working; the diff must not assume one platform. - FLAG — CUDA/Jetson/ONNX-provider introspection changes are correct on every runtime, not just the author's box (one-runtime introspection bugs mis-route model loading).
- NIT — restriction present but mis-scoped or wrong severity (HARD where SOFT fits, missing
applies_to_runtimes).
Not blocking
- Do NOT demand a hosted-platform test for a change with no observable hosted/serverless behavior difference (pure local refactor, docstrings, typing). Hosted coverage under
tests/inference/hosted_platform_tests/is required only when the diff changes what a request returns on lambda / serverless / dedicated. - Do NOT demand a restriction for a block that is genuinely runtime-agnostic (stateless, no network, no LAN, no GPU-handle assumption).
- Do NOT demand a flip-the-flag-both-ways test for env vars that only tune values (URLs, timeouts) without forking code paths.
Standards
-
Execution-mode dispatch is exhaustive.
run()branches onself._step_execution_modeand terminates inelse: raise ValueError(f"Unknown step execution mode: ..."). Canonical shape:inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py—run()branchesStepExecutionMode.LOCAL/StepExecutionMode.REMOTE,run_locallyviaself._model_manager,run_remotelyviaInferenceHTTPClienthonoringWORKFLOWS_REMOTE_API_TARGET/LOCAL_INFERENCE_API_URL. (StepExecutionModeis defined ininference/core/workflows/prototypes/block.pyand re-exported fromcore_steps/common/entities.py.) -
Local and remote paths stay semantically equivalent.
run_locallyandrun_remotelyaccept the same params and return the same shape; every manifest field is forwarded on both paths (one-path param wiring is a silent divergence). -
Metadata must match behavior. A block's
get_restrictions()/get_air_gapped_availability()declarations are consumed by the describe endpoint and the platform UI to decide where a block may run. The declaration and the actualrun()guard must agree — a code path that raises for a runtime needs aRuntimeRestrictionfor it, and vice-versa. LAN/same-process shape:onvif_movement/v1.py(HARD +StepExecutionMode.REMOTE, plus HARD +Runtime.HOSTED_SERVERLESS/Runtime.DEDICATED_DEPLOYMENTbecause hosted can't reach a customer LAN). -
Runtime routing has no orphaned path. A new backend, env flag, or hardware gate leaves BOTH the on and off value working and tested; the legacy path (ONNX registry entries) stays intact. Backend routing lives in
inference/core/models/inference_models_adapters.py+inference/core/env.py(USE_INFERENCE_MODELS,_PLATFORM_SPECIFIC_USE_INFERENCE_MODELS_DEFAULT,DISABLED_INFERENCE_MODELS_BACKENDS). Rollout history: #2096 added per-deploymentDISABLED_INFERENCE_MODELS_BACKENDS; #2128 added missing registry entries so RFDETR-NAS works under inference-models; #2136 reverted making inference-models the default backend (the default flip is contentious — review carefully). -
Per-platform introspection is correct on every runtime. CUDA-version / Jetson / ONNX-provider detection that gates model loading must hold across platforms, not just the author's (CUDA acquisition fixed in #2515; Jetson environment detection in runtime introspection + dockerfiles fixed in #1796).
-
Hosted-only code paths get hosted coverage. Behavior that differs on lambda / serverless / dedicated is exercised in
tests/inference/hosted_platform_tests/(parametrized overPlatformEnvironmentviaconftest.py; seetest_workflows.py,test_roboflow_models.py). Hosted-only plumbing regresses silently under local-only tests (remote cold-start header aggregation reverted in #2222;ROBOFLOW_ENVIRONMENTpropagation to the modal webrtc worker fixed in #2179; dedicatedhosted=True/DEDICATED_DEPLOYMENT_IDusage attribution fixed in #817).
Key files & Reference PRs
Files:
inference/core/workflows/prototypes/block.py— the contract:StepExecutionMode,Runtime,Severity,RuntimeRestriction, presets (STATEFUL_VIDEO_HTTP_SOFT_RESTRICTION,COOLDOWN_HTTP_SOFT_RESTRICTION), andget_restrictions/get_air_gapped_availability/get_execution_engine_compatibilityhooks. Read first.inference/core/workflows/core_steps/models/roboflow/object_detection/v1.py— canonical LOCAL/REMOTE dispatch.inference/core/workflows/core_steps/sinks/onvif_movement/v1.py— same-process + LAN block done right (two HARD restrictions).inference/core/models/inference_models_adapters.py+inference/core/env.py— backend routing with per-platform default and per-deployment disable.tests/inference/hosted_platform_tests/conftest.py(+test_workflows.py,test_roboflow_models.py) — hosted/serverless/localhost parametrization.
Reference PRs: #2096, #2128, #2136 (backend rollout / default flip), #2515, #1796 (per-platform introspection), #2222, #2179, #817 (hosted-only plumbing).