Phoenix Arize Setup

DevOps & Security

phoenix-arize-setup

Phoenix Arize Setup is an Agent Skill for devops & security published in the a5c-ai/babysitter repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Arize Phoenix observability platform setup for LLM debugging and evaluation

View on GitHub ↗

QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

Open your project in Codex.
Copy and paste the prompt below.
Review files and risks before approving any action.

Prompt to paste

Inspect the Agent Skill at https://github.com/a5c-ai/babysitter/blob/HEAD/library/specializations/ai-agents-conversational/skills/phoenix-arize-setup/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/phoenix-arize-setup, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Phoenix Arize Setup Skill

Capabilities

Set up Phoenix local server
Configure tracing instrumentation
Design evaluation experiments
Implement embedding visualizations
Set up retrieval analysis
Create custom evaluations with LLM-as-judge

Target Processes

llm-observability-monitoring
agent-evaluation-framework

Implementation Details

Core Features

Tracing: OpenTelemetry-based LLM traces
Evals: LLM-as-judge evaluations
Embeddings: Visualization and drift detection
Retrieval: RAG quality analysis
Datasets: Experiment management

Instrumentation

OpenAI auto-instrumentation
LangChain instrumentation
LlamaIndex instrumentation
Custom span creation

Configuration Options

Phoenix server setup
Trace sampling
Evaluation metrics
Embedding models
Export settings

Best Practices

Comprehensive instrumentation
Regular evaluation runs
Monitor embedding drift
Analyze retrieval quality

Dependencies

arize-phoenix
openinference-instrumentation-openai