orchestration-authoring

Orchestra Orchestration Authoring Reference

Complete reference for creating valid, idiomatic Orchestra orchestration files.

Detailed references (read on demand):

• references/examples.md -- Full real-world example orchestrations
• references/full-schema-reference.md -- Complete property-level documentation with all edge cases

Format

Orchestrations are single JSON or YAML objects. Three fields are required: name, description, steps. YAML is recommended for orchestrations with multi-line prompts (use | block scalars).

Editor schema validation. Bind the orchestration JSON Schema for autocomplete, type-checking, and unknown-field errors. Pick one of three options based on how you obtained Orchestra:

1. Public URL (works anywhere, requires network the first time):

   • JSON: "$schema": "https://raw.githubusercontent.com/MoaidHathot/orchestra/main/schemas/orchestration.schema.json"
   • YAML: # yaml-language-server: $schema=https://raw.githubusercontent.com/MoaidHathot/orchestra/main/schemas/orchestration.schema.json
   • Same pattern for orchestra.mcp.schema.json and orchestra.services.schema.json.
   • For version-pinned validation, replace main with a release tag (e.g., v0.2.0).

2. Local copy bundled with the tool (offline, version-pinned to your installed Orchestra):

   • Run once in your project root: orchestra schemas
   • This writes the three schemas to ./.orchestra/schemas/.
   • Then reference them with a relative path:
     • JSON: "$schema": ".orchestra/schemas/orchestration.schema.json"
     • YAML: # yaml-language-server: $schema=./.orchestra/schemas/orchestration.schema.json
   • Use --output <dir> to choose a different folder; --force to overwrite.

3. Repository-relative path (only inside this repository's examples/ folder):

   • JSON: "$schema": "../schemas/orchestration.schema.json"
   • YAML: # yaml-language-server: $schema=../schemas/orchestration.schema.json

YAML modelines work in VS Code (Red Hat YAML extension), JetBrains IDEs, and any editor on yaml-language-server. A top-level $schema: key is also supported.

Top-Level Properties

Typed Inputs (InputDefinition)

Each key in inputs is the input name. Values:

Hooks

Hooks run after step or orchestration lifecycle events. They are top-level orchestration configuration, not DAG steps. Use them for follow-up automation such as notifications, archival, incident creation, or failure triage.

Hook Filters (when)

In v1, filtering is intentionally small and step-focused:

when:
  steps:
    names: [build, deploy]
    status: failed
    match: any

when.steps properties:

Hook Payload (payload)

Hook payloads are structured JSON sent to the action on stdin.

Hook Action (action)

In v1, hooks support script actions only.

*Use exactly one of script or scriptFile.

Example:

hooks:
  - name: archive-run-failure
    on: orchestration.failure
    payload:
      detail: compact
      steps: failed
      includeRefs: true
    action:
      type: script
      shell: pwsh
      scriptFile: ./hooks/archive-failure.ps1

Hooks are different from Prompt step handlers:

• inputHandlerPrompt and outputHandlerPrompt transform Prompt step input/output
• hooks run after lifecycle events and do not alter orchestration execution

Steps

Steps form a DAG. Steps with no dependsOn run first (in parallel). Downstream steps run when all dependencies complete.

Base Step Properties (all types)

Prompt Step (type: "Prompt")

Calls an LLM.

*Mutual exclusion: use systemPrompt OR systemPromptFile, not both. Same for userPrompt/userPromptFile, inputHandlerPrompt/inputHandlerPromptFile, outputHandlerPrompt/outputHandlerPromptFile.

System Prompt Modes

• append (default): Your system prompt is added to the SDK's built-in prompts, preserving coding capabilities.
• replace: Your system prompt completely replaces the SDK's built-in prompts.
• customize: Selectively override individual sections of the built-in prompt. Use with systemPromptSections.

System Prompt Section Overrides (systemPromptSections)

Used with systemPromptMode: "customize". Keys are section identifiers, values are override objects:

Each section override:

Infinite Sessions (infiniteSessions)

Controls automatic context compaction for long-running steps.

Image Attachments (attachments)

Send images to the LLM alongside the prompt for vision/analysis tasks.

Each attachment has a type:

File attachment (type: "file"): reads an image from disk.

Blob attachment (type: "blob"): inline base64-encoded image data.

Model Tuning, Working Directory & Authentication (Copilot)

Per-step Copilot controls, all opt-in (omit to inherit the host/provider default):

• reasoningSummary — none / concise / detailed. Verbosity of the model's reasoning summary (distinct from reasoningLevel, the reasoning effort).
• contextTier — default / longContext. Opts into the model's extended context window where supported.
• workingDirectory — the agent's working directory for shell/file tools and config discovery (custom instructions, .github/agents, .github/mcp.json). Template-resolved ({{param.*}}/{{env.*}}/{{vars.*}}) and validated to exist at run time.
• githubToken — authenticates this step's Copilot session, overriding the host default. Prefer {{env.GITHUB_TOKEN}} over a literal secret. Host-level default: orchestra.json copilot.gitHubToken / copilot.useLoggedInUser.

Human-in-the-Loop (humanInput)

humanInput: true routes the agent's elicitation and exit-plan-mode (plan approval) requests to Orchestra's human-in-the-loop — the same pending-input surface as the Approval step and orchestra_request_user_input (answer via POST /api/orchestrations/{name}/runs/{runId}/respond). Session-bound (does NOT survive a host restart). Default (off) resolves them autonomously. Pairs naturally with permissionPolicy: requireHumanApproval.

Permission Policy (permissionPolicy)

Controls how the agent's permission requests (shell, file read/write, url, mcp, …) are resolved.

{ "permissionPolicy": { "mode": "denyList", "deny": ["shell", "url", "*.env"] } }

Sandbox (sandbox)

Constrains the agent's shell/file/network tool access, applied to the live session via the runtime's options-update RPC. Sandbox paths are passed verbatim (not template-resolved); enforcement is provided on Linux/macOS.

{
  "sandbox": {
    "enabled": true,
    "filesystem": { "readonly": ["/work/repo"], "denied": ["/etc"] },
    "network": { "allowOutbound": false }
  }
}

Http Step (type: "Http")

Makes an HTTP request, captures response body.

Transform Step (type: "Transform")

Pure string interpolation (no LLM, no I/O).

Command Step (type: "Command")

Executes a direct executable as a child process, captures stdout. Use this for commands such as dotnet, git, dnx, or npx.

Do not use Command for shell snippets or wrappers such as pwsh -Command, powershell -Command, bash -c, or sh -c. Use a Script step instead so quoting, pipes, multiline logic, and JSON values are handled by the script file invocation.

Script Step (type: "Script")

Executes an inline or file-based script via a shell interpreter (e.g., pwsh, bash, python, node). Captures stdout. Use this for shell snippets, pipelines, multi-line scripts, quoting-sensitive values, JSON manipulation, and anything that would otherwise be passed to pwsh -Command or bash -c. Use YAML | blocks for best readability.

*Mutual exclusion: use script OR scriptFile, not both. scriptFile paths resolve relative to the orchestration file's directory.

Pass values into scripts with arguments or stdin instead of interpolating large or heavily quoted values into the script body. In PowerShell, arguments are available as $args[0], $args[1], and so on.

For file paths that should be relative to the orchestration file, anchor them explicitly with {{orchestration.sourceDirectory}}. Do not pass bare relative paths to runtime file-writing code, because process working directories can differ between hosts.

Orchestration Step (type: "Orchestration")

Invokes another registered orchestration. Use this when a flow should delegate to a reusable child orchestration instead of duplicating its steps.

mode is sync or async. In sync mode, the parent waits for the child to finish and uses the child's final output as this step's output. In async mode, the parent continues after dispatch.

parameters maps child input names to values. Values support template expressions and are passed as strings at runtime.

inputHandlerPrompt can reshape child parameters before launch. It must return a JSON object mapping parameter names to string values. If handler parsing fails, runtime falls back to the original parameters, so use Script validation for hard guarantees.

Drill-in template bindings. For every step whose type is Orchestration, dependants can read the child run's per-step data via these template accessors. The data is populated on every terminal branch (success, failure, cancellation) and is in-process / untruncated — no MCP round-trip needed.

Use these for self-healing repair patterns: a downstream Prompt step can inspect {{attempt-1.steps.build.error}} and {{attempt-1.steps.codegen.output}} to build a corrective prompt — works whether attempt-1 succeeded, failed, or was cancelled. See examples/self-healing-with-child-bindings.yaml for a complete pattern, or docs/orchestration-step-deep-dive.md for the full reference.

Approval Step (type: "Approval")

Pauses the orchestration and waits for human input. The step persists a pending input record, transitions to AwaitingInput status, fires the step.awaitingInput hook event, and blocks until a user responds via the host's HumanInput API (or via the CLI / Portal). The user's response (reply or choice, with reply winning) becomes the step's output content and can be referenced by downstream steps via {{stepName.output}}.

Approval steps survive host restarts: the persisted record is preserved, and on resume the step re-attaches to the still-outstanding wait.

Example:

- name: review-deploy
  type: Approval
  dependsOn: [build]
  prompt: "Approve deploy of {{param.service}} to {{param.env}}? Build: {{build.output}}"
  choices: [approve, reject]

Respond via API or CLI:

orchestra pending
orchestra respond <orchestration-name> <runId> review-deploy --choice approve --by alice

Or via raw HTTP:

POST /api/orchestrations/<orchestration-name>/runs/<runId>/respond?step=review-deploy
{ "choice": "approve", "respondedBy": "alice" }

Engine-Tool HITL Variant: orchestra_request_user_input

For LLM-decided "ask the human only when needed" pauses inside Prompt steps, opt the Prompt step into the request_user_input engine tool. The agent can then call orchestra_request_user_input(prompt, choices?) mid-conversation; the call blocks until the user responds, and the reply is returned as the tool result so the agent continues with the answer in hand.

- name: writer
  type: Prompt
  systemPrompt: |
    You write articles. Use orchestra_request_user_input ONLY when the topic is
    genuinely ambiguous and a clarifying decision would meaningfully improve the
    output. Otherwise, just write the article.
  userPrompt: "Write an article about {{param.topic}}."
  model: claude-opus-4.6
  enableTools: [request_user_input]

Differences from the declarative Approval step:

Both paths emit the step.awaitingInput hook event with the same payload structure, both persist a PendingInputRecord, and both route through POST /api/orchestrations/{name}/runs/{runId}/respond?step={stepName}.

Loop Configuration (Checker Pattern)

A Prompt step with loop acts as a checker for iterative refinement.

The checker evaluates the target's output. If exitPattern is NOT found (case-insensitive), the target re-runs with checker feedback. Repeats up to maxIterations.

Subagents

Multi-agent delegation within a single Prompt step.

*Exactly one of prompt or promptFile required.

Retry Policy

Triggers

Manual (default)

trigger:
  type: manual

Scheduler

Loop Trigger

Webhook

WebhookResponseConfig: waitForResult (bool), responseTemplate (string), timeoutSeconds (int, default 120).

All triggers share: type (required), enabled (bool, default true), inputHandlerPrompt (string), inputHandlerModel (string).

MCP Definitions

Local MCP (stdio transport)

mcps:
  - name: filesystem
    type: local
    command: npx
    arguments:
      - "-y"
      - "@modelcontextprotocol/server-filesystem"
      - "{{workingDirectory}}"

Remote MCP (HTTP transport)

mcps:
  - name: cloud-tools
    type: remote
    endpoint: "https://mcp.example.com/tools"
    headers:
      Authorization: "Bearer {{env.TOKEN}}"

MCPs defined at orchestration level. Steps reference by name: mcps: [filesystem]. A companion mcp.json or orchestra.mcp.json file can define MCPs externally.

Template Expressions

Syntax: {{expression}} -- supported in prompts, URLs, headers, bodies, templates, command arguments, working directories, environment values, stdin, variable values, MCP configs, skill directory paths.

Orchestration-step accessors (only on steps whose type is Orchestration):

Engine Tools (Built-in, available to all Prompt steps)

Opt-in Engine Tools (per-step or default)

These tools must be explicitly enabled via enableTools on a Prompt step (or defaultEnableTools at the orchestration level). Existing pipelines see no behavior change.

Common Patterns

1. Fan-Out / Fan-In

Multiple root steps (no dependsOn) run in parallel; a downstream step depends on all of them to synthesize results.

defaultModel: claude-opus-4.6
steps:
  - name: research-a
    type: Prompt
    systemPrompt: Research topic A.
    userPrompt: "{{param.topic}}"
  - name: research-b
    type: Prompt
    systemPrompt: Research topic B.
    userPrompt: "{{param.topic}}"
  - name: synthesize
    type: Prompt
    dependsOn: [research-a, research-b]
    systemPrompt: Synthesize the research.
    userPrompt: |
      Research A: {{research-a.output}}
      Research B: {{research-b.output}}

2. Loop/Checker (Iterative Refinement)

A checker step loops a target step until quality is met.

- name: review
  type: Prompt
  dependsOn: [write-draft]
  systemPrompt: Review. Say APPROVED if good, REVISE if not.
  userPrompt: "{{write-draft.output}}"
  loop:
    target: write-draft
    maxIterations: 3
    exitPattern: APPROVED

3. Subagent Delegation

Coordinator delegates to specialized subagents.

- name: coordinator
  type: Prompt
  systemPrompt: Delegate to your specialists.
  userPrompt: "{{param.task}}"
  subagents:
    - name: researcher
      description: Finds facts from the web.
      prompt: You are a researcher.
      mcps: [web-fetch]
      infer: true
    - name: writer
      description: Writes polished content.
      prompt: You are a writer.
      infer: true

4. Gate / Early Exit

A step checks conditions and halts the orchestration if nothing to do.

- name: gate
  type: Prompt
  systemPrompt: |
    Check if there are incidents.
    If none, call orchestra_complete.
    If there are, list them.
  userPrompt: "{{check-incidents.output}}"

5. Input/Output Handlers

Pre-process dependency outputs or post-process LLM output.

- name: analyze
  type: Prompt
  dependsOn: [fetch-data]
  inputHandlerPrompt: Extract only numeric data points from the input.
  outputHandlerPrompt: Format as a markdown table.
  systemPrompt: Analyze the data.
  userPrompt: "{{fetch-data.output}}"

6. Multi-Step Pipeline (all 5 step types)

Command -> Script -> Prompt -> Transform -> Http -> Orchestration

defaultModel: claude-opus-4.6
steps:
  - name: build
    type: Command
    command: dotnet
    arguments: [build]
  - name: gather-info
    type: Script
    dependsOn: [build]
    shell: pwsh
    script: |
      Get-ChildItem bin -Recurse -Filter '*.dll' |
        Select-Object -ExpandProperty Name |
        ConvertTo-Json
  - name: analyze
    type: Prompt
    dependsOn: [build, gather-info]
    systemPrompt: Analyze build output.
    userPrompt: |
      Build: {{build.output}}
      Artifacts: {{gather-info.output}}
  - name: report
    type: Transform
    dependsOn: [analyze]
    template: |
      # Report
      {{analyze.output}}
  - name: notify
    type: Http
    dependsOn: [report]
    method: POST
    url: "{{vars.webhookUrl}}"
    body: '{"text": "{{report.output}}"}'

7. Webhook with Input Handler

Normalize arbitrary payloads into expected parameters.

trigger:
  type: webhook
  maxConcurrent: 5
  inputHandlerPrompt: Extract 'eventType' and 'data' from the JSON payload.

8. Scheduled Monitoring

Run on interval, gate on no-action.

trigger:
  type: scheduler
  intervalSeconds: 300

9. Cross-Step File References

Steps save files, downstream steps reference them.

- name: consumer
  type: Prompt
  dependsOn: [producer]
  systemPrompt: Read and analyze the saved files.
  userPrompt: |
    Files: {{producer.files}}
    First file: {{producer.files[0]}}

10. Variables with Recursive Expansion

variables:
  appName: my-app
  registry: "{{env.CONTAINER_REGISTRY}}/{{vars.appName}}"
  artifactPath: "/artifacts/{{vars.appName}}/{{orchestration.runId}}"

11. Customize System Prompt with Section Overrides

Surgically control specific sections while preserving others.

- name: code-review
  type: Prompt
  systemPrompt: Review the code for accessibility.
  systemPromptMode: customize
  systemPromptSections:
    tone:
      action: replace
      content: Be direct and structured.
    code_change_rules:
      action: remove  # Read-only, no modifications
    guidelines:
      action: append
      content: |
        - Follow WCAG 2.1 AA guidelines.
        - Flag contrast ratio violations.
  userPrompt: "{{code-step.output}}"

12. Image Attachments for Vision Analysis

Send images from files or prior step output.

- name: analyze-screenshot
  type: Prompt
  systemPrompt: Analyze this UI for accessibility issues.
  userPrompt: Describe what you see and identify problems.
  attachments:
    - type: file
      path: "{{param.imagePath}}"
      displayName: "Screenshot"

13. Infinite Sessions for Long-Running Tasks

Control context compaction thresholds per step.

- name: large-refactor
  type: Prompt
  systemPrompt: Refactor the entire module.
  userPrompt: "{{gather-code.output}}"
  infiniteSessions:
    enabled: true
    backgroundCompactionThreshold: 0.85
    bufferExhaustionThreshold: 0.97

14. Human-in-the-Loop Approval Gate (Declarative)

A deploy that pauses for a human reviewer; the response feeds the next step.

defaultModel: claude-opus-4.6
inputs:
  service: { type: string, required: true }
  env: { type: string, enum: [staging, production], required: true }
steps:
  - name: build
    type: Command
    command: dotnet
    arguments: [build]

  - name: review-deploy
    type: Approval
    dependsOn: [build]
    prompt: |
      Approve deploy of {{param.service}} to {{param.env}}?

      {{build.output}}
    choices: [approve, reject]
    # No timeoutSeconds — wait indefinitely. Orchestration timeout is paused
    # while awaiting input by default (pauseTimeoutDuringWait: true).

  - name: announce
    type: Transform
    dependsOn: [review-deploy]
    template: "Decision: {{review-deploy.output}}"

15. LLM-Decided HITL (Engine Tool)

The agent only pauses if it needs clarification. Existing pipelines without enableTools are unaffected.

- name: writer
  type: Prompt
  systemPrompt: |
    You write articles. Use orchestra_request_user_input ONLY when the topic
    is genuinely ambiguous and a clarifying decision from the user would
    meaningfully improve the output. Otherwise just write the article.
  userPrompt: "Write a 200-word article about {{param.topic}}."
  model: claude-opus-4.6
  enableTools: [request_user_input]

16. Notify on HITL Pause via Hook

Wire step.awaitingInput into a script hook that posts to Slack/Teams/email.

hooks:
  - name: slack-on-pause
    on: step.awaitingInput
    payload:
      detail: compact
      includeRefs: true
    action:
      type: script
      shell: pwsh
      script: |
        $payload = $input | Out-String | ConvertFrom-Json
        $body = @{
          text = "[$($payload.orchestration.name)] needs input on '$($payload.step.name)'"
        } | ConvertTo-Json
        Invoke-RestMethod -Uri $env:SLACK_WEBHOOK -Method Post -Body $body -ContentType 'application/json'

17. Approval with Timeout Fallback

Auto-acknowledge a low-priority alert if no one responds within 5 minutes.

- name: triage
  type: Approval
  prompt: "Acknowledge incident {{param.incidentId}}? (auto-acknowledge in 5m)"
  choices: [acknowledge, escalate]
  timeoutSeconds: 300
  onTimeout: defaultResponse
  defaultResponse: acknowledge

18. Governed Prompt Step (Permission Policy + Sandbox + HITL)

Constrain a read-only review agent: deny shell/url tools, sandbox the filesystem and network, and route any elicitation/plan-approval to a human. All controls are opt-in.

- name: review
  type: Prompt
  systemPrompt: |
    You are a read-only reviewer. Do not run shell commands, fetch URLs, or modify
    files. Ask the operator if you need a decision.
  userPrompt: "{{param.question}}"
  model: claude-opus-4.6
  workingDirectory: /work/repo
  humanInput: true
  permissionPolicy:
    mode: denyList
    deny: [shell, url]
  sandbox:
    enabled: true
    filesystem:
      readonly: [/work/repo]
      denied: [/etc]
    network:
      allowOutbound: false

Registering Orchestrations

Orchestrations can be registered in Orchestra via:

1. REST API POST /api/orchestrations/json with body { "json": "<orchestration JSON string>" } -- registers from raw JSON content.
2. REST API POST /api/orchestrations with body { "paths": ["<file path>"] } -- registers from file path.
3. MCP Control Plane register_orchestration tool -- registers from file path.
4. Directory Scan -- Orchestra can auto-scan a directory on startup.

Common Mistakes to Avoid

1. Do NOT invent properties. Only use properties documented above. There is no if, condition, forEach, parallel, or output property.
2. Do NOT use systemPrompt AND systemPromptFile together. They are mutually exclusive. Same for all *File pairs (including script/scriptFile).
3. Loop target must be a dependency. The checker step must have the target in its dependsOn.
4. Step names must be unique within the orchestration.
5. No circular dependencies. The DAG must be acyclic.
6. parameters is a string array of names, not key-value pairs. Values come at runtime.
7. Model is required for Prompt steps unless defaultModel is set at the orchestration level. Use "claude-opus-4.6" as default.
8. Template expressions are {{...}}, not ${...} or {...}.
9. Boolean/Number inputs: values are always strings in JSON. "true", "false", "42".
10. dependsOn references step names, not types or indices.
11. mcps on steps is an array of strings (MCP names), not MCP definition objects.
12. Script steps require shell. It has no default -- always specify it (e.g., "pwsh", "bash").
13. Do NOT use Command with pwsh -Command, powershell -Command, or bash -c for script logic. Use type: Script, shell: pwsh, and script: | instead.
14. Do NOT rely on the host process working directory for runtime file paths. Use {{orchestration.sourceDirectory}} to build absolute paths relative to the orchestration file.
15. systemPromptSections requires systemPromptMode: "customize". Section overrides are ignored with append or replace.
16. Image attachments require a vision-capable model (e.g., claude-opus-4.6, gpt-4o). Non-vision models will not understand the images.
17. infiniteSessions thresholds are ratios (0.0-1.0), not token counts. 0.80 means 80% of context used.
18. hooks is a top-level array, not a step-level property.
19. Hook actions require exactly one of script or scriptFile. Do not specify both.
20. Hook failurePolicy only supports warn or ignore in v1.
21. Approval steps require prompt. Without it, parsing fails.
22. onTimeout: defaultResponse requires a defaultResponse value. Parsing fails otherwise.
23. enableTools only accepts opt-in tool names (currently just "request_user_input"). Always-on tools (orchestra_set_status, orchestra_complete, file save/read) are not listed here.
24. orchestra_request_user_input does NOT survive host restarts. Agent sessions are volatile. Use the declarative Approval step for long-lived gates that must endure restarts.
25. Approval and engine-tool waits both fire step.awaitingInput — use a single hook to handle notifications for both paths.
26. humanInput waits do NOT survive host restarts. Like orchestra_request_user_input, elicitation/plan-approval pauses are session-bound. Use the declarative Approval step for restart-durable gates.
27. permissionPolicy.deny is only used in denyList mode. In approveAll (default) and requireHumanApproval modes the deny list is ignored.
28. Sandbox paths are passed verbatim (NOT template-resolved), and sandbox enforcement is provided on Linux/macOS. By contrast, workingDirectory and githubToken ARE template-resolved via {{env.*}}/{{param.*}}/{{vars.*}}.
29. Per-step Copilot controls are opt-in. reasoningSummary, contextTier, workingDirectory, githubToken, humanInput, permissionPolicy, and sandbox all default to off/unset — existing orchestrations are unchanged.

Naming Conventions

• Orchestration names: kebab-case (my-deployment-pipeline)
• Step names: kebab-case (build-artifact, security-scan)
• Variable names: camelCase (appName, slackWebhookUrl)
• Input names: camelCase (serviceName, dryRun)