Tts Integration
Developmenttts-integrationTts Integration is an Agent Skill for development published in the rapidaai/voice-ai repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Add or modify text-to-speech providers in assistant-api with transport-aware behavior (WS/SSE/SDK/HTTP), correct packet lifecycle, and UI/provider wiring.
QUICK START
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/rapidaai/voice-ai/blob/HEAD/.claude/skills/tts-integration/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/tts-integration, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
TTS Integration Skill
Mission
Integrate TTS providers with correct streaming/flush/interruption semantics and stable tts_latency_ms metrics.
Inputs expected from user
- Transport model: bidirectional WebSocket, SSE/chunked HTTP, SDK callback, or request-response flush.
- Voice/model parameter requirements.
- Output audio format from provider.
If user does not answer:
- Select nearest baseline transport and preserve existing packet semantics.
- Keep internal output format compatibility with existing streamer path.
Hard boundaries
In scope:
api/assistant-api/internal/transformer/<provider>/tts.go(+ provider option/normalizer helpers)api/assistant-api/internal/transformer/transformer.go(factory case)- optional contract updates:
api/assistant-api/internal/type/tts_transformer.go,packet.go - provider config JSON + UI component files for TTS
Out of scope:
- STT-only logic changes unrelated to shared provider setup
- telephony transport internals
- EOS/VAD internals
Transport mapping
- WS streaming baseline:
deepgram,rime,sarvam - SSE/chunked baseline:
minimax - SDK/API callback baseline:
azure,google - HTTP flush baseline:
aws,resembleai
Packet contract
Input packets:
LLMResponseDeltaPacketLLMResponseDonePacketInterruptionPacket
Required outputs:
TextToSpeechAudioPacketTextToSpeechEndPacketConversationEventPacket{Name:"tts", ...}MessageMetricPacket{Name:"tts_latency_ms"}once per utterance
Implementation workflow
- Pick transport baseline and clone lifecycle shape.
- Implement provider under
transformer/<provider>/. - Add factory registration in
transformer.go. - Wire UI metadata (
voices,languages,text-to-speech-models) and config form. - Verify interruption behavior clears output state without deadlocks.
- Add provider tests (init, streaming, completion, interruption, metric-on-first-audio).
Validation commands
go test ./api/assistant-api/internal/transformer/... -run TestTTSgo test ./api/assistant-api/internal/transformer/<provider>/...cd ui && yarn test providers./.claude/skills/tts-integration/scripts/validate.sh --check-diff --provider <provider>