OpenParable
Skip to content
← Back to skills

Llmobs Integrations

Development
llmobs-integrations

Llmobs Integrations is an Agent Skill for development published in the DataDog/dd-trace-py repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

dd-trace-py LLMObs integration development guide. Use when creating, modifying, or debugging LLMObs integrations for LLM/AI libraries in the Python tracer. Covers BaseLLMIntegration, stream handling, message extraction, token counting, tool call parsing, and VCR-based testing patterns. Triggers: "llmobs", "LLMObs", "BaseLLMIntegration", "llmobs_set_tags", "_llmobs_set_tags", "BaseStreamHandler", "submit_to_llmobs", "integration.trace", "LLM span", "VCR", "cassette", "anthropic", "openai", "google_genai", "claude_agent_sdk", "generative-ai", "LLM integration", "llmobs_enabled".

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/DataDog/dd-trace-py/blob/HEAD/.claude/skills/llmobs-integrations/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/llmobs-integrations, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

dd-trace-py LLMObs Integrations

LLMObs integrations enable Datadog LLM Observability for AI/LLM libraries. They extract model inputs, outputs, token usage, and tool calls from traced spans. This skill should be used in addition to the apm-integrations skill.

Two-Layer Architecture

LLMObs integrations consist of two cooperating layers:

  1. Patch Layer (ddtrace/contrib/internal/{name}/patch.py) -- wraps library functions. Standard request/response LLM integrations construct LlmRequestEvent and use core.context_with_event() so the LLM tracing subscriber owns span lifecycle and LLMObs tag extraction.
  2. Integration Layer (ddtrace/llmobs/_integrations/{name}.py) -- extends BaseLLMIntegration, implements _set_base_span_tags() and _llmobs_set_tags() to extract and set provider-specific messages, tools, metadata, and token metrics.

Both layers must work together. The patch layer identifies the operation and passes request/response data through the event; the integration layer controls what data is extracted.

Active Patch Patterns

  • Event-based request spans: Use LlmRequestEvent with core.context_with_event() for new standard request/response LLM integrations. Anthropic is the canonical reference. This is the preferred pattern.
  • Direct integration spans: Some existing or specialized integrations still call integration.trace() and integration.llmobs_set_tags() directly, especially for child spans, agent/tool spans, or integrations not yet migrated. Google GenAI, OpenAI tool spans, and Claude Agent SDK are useful references.

Key Files

PurposeFile
Base LLM integration classddtrace/llmobs/_integrations/base.py (BaseLLMIntegration)
Stream handler base classesddtrace/llmobs/_integrations/base_stream_handler.py (BaseStreamHandler, StreamHandler, AsyncStreamHandler)
Shared utilitiesddtrace/llmobs/_integrations/utils.py
LLMObs annotation helperddtrace/llmobs/_utils.py (_annotate_llmobs_span_data)
LLMObs constantsddtrace/llmobs/_constants.py
LLMObs typesddtrace/llmobs/types.py (Message, AudioPart, ToolCall, ToolResult, ToolDefinition)
Integration registryddtrace/llmobs/_integrations/__init__.py

Reference Integrations

Always read 1-2 references before writing or modifying LLMObs code.

ProviderPatch FileLLMObs IntegrationLLMObs Tests
Anthropic (canonical)ddtrace/contrib/internal/anthropic/patch.pyddtrace/llmobs/_integrations/anthropic.pytests/contrib/anthropic/test_anthropic_llmobs.py
Claude Agent SDK (latest, agent pattern)ddtrace/contrib/internal/claude_agent_sdk/patch.pyddtrace/llmobs/_integrations/claude_agent_sdk.pytests/contrib/claude_agent_sdk/test_claude_agent_sdk_llmobs.py
OpenAIddtrace/contrib/internal/openai/patch.pyddtrace/llmobs/_integrations/openai.pytests/contrib/openai/test_openai_llmobs.py
Google GenAIddtrace/contrib/internal/google_genai/patch.pyddtrace/llmobs/_integrations/google_genai.pytests/contrib/google_genai/test_google_genai_llmobs.py

Use Anthropic as the canonical reference for standard LLM integrations. Use Claude Agent SDK for agent-pattern integrations (agent spans, tool child spans, thinking blocks).

Abstract Methods to Implement

Subclass BaseLLMIntegration and implement:

_set_base_span_tags(span, **kwargs)

Set provider-specific APM tags on the span (e.g., {name}.request.model).

_llmobs_set_tags(span, args, kwargs, response, operation)

Extract and annotate all LLMObs fields on the span:

FieldDescription
kind"llm" for LLM calls, "agent" for agent calls, "tool" for tool calls
model_nameModel identifier (e.g., "claude-3-sonnet-20240229")
model_providerProvider name (e.g., "anthropic", "openai")
input_messagesList of Message objects from request
output_messagesList of Message objects from response
metadataDict of sanitized request parameters (temperature, top_p, etc.)
metricsToken usage dict with INPUT_TOKENS_METRIC_KEY, OUTPUT_TOKENS_METRIC_KEY, TOTAL_TOKENS_METRIC_KEY
tool_definitionsList of ToolDefinition objects if tools are passed

Fields are usually set via _annotate_llmobs_span_data(...), not raw span._set_ctx_items(...).

Key Constraints

  • submit_to_llmobs=True must be set on LlmRequestEvent for event-based request spans or passed to integration.trace() for direct LLMObs spans
  • ctx.dispatch_ended_event() must run on success and error paths for event-based patch wrappers
  • Streaming must use BaseStreamHandler/AsyncStreamHandler -- never consume streams directly
  • Event-based patch wrappers should not call span.set_exc_info(), span.finish(), or integration.llmobs_set_tags() directly; the tracing subscriber handles that when the event ends
  • Direct integration spans must keep integration.llmobs_set_tags() and span lifecycle handling aligned with the closest current reference
  • Integration instance must be stored on the module: module._datadog_integration = MyLibIntegration(integration_config=config.mylib)

Message Types

from ddtrace.llmobs.types import AudioPart, Message, ToolCall, ToolResult, ToolDefinition

# Input/output messages
Message(content="text", role="user")
Message(content="response", role="assistant", tool_calls=[...])

# Audio attachments in multimodal messages
AudioPart(mime_type="audio/wav", content="<base64-audio>")
Message(content="", role="user", audio_parts=[...])

# Tool calls (in output messages)
ToolCall(name="get_weather", arguments={"city": "NYC"}, tool_id="toolu_123", type="tool")

# Tool results (in input messages)
ToolResult(result="72F sunny", tool_id="toolu_123", type="tool_result")

# Tool definitions (from request parameters)
ToolDefinition(name="get_weather", description="...", schema={...})

Debugging Quick Tips

  • No LLMObs spans -- check submit_to_llmobs=True, ctx.dispatch_ended_event(), and llmobs_enabled
  • Wrong messages -- check message extraction handles multi-part content and tool blocks
  • Wrong tokens -- check field name mapping (libraries use different names for token counts)
  • Streaming broken -- verify BaseStreamHandler subclass, check finalize_stream() dispatches the ended event or finishes direct-trace spans according to the reference pattern
  • DD_TRACE_DEBUG=true to see patching activity and span creation

See Failure Modes for detailed debugging guide.

Reference Files