Stt Integration
Developmentstt-integrationStt Integration is an Agent Skill for development published in the rapidaai/voice-ai repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Add or modify speech-to-text providers in assistant-api with transport-aware ingestion (WS/SDK/HTTP), transcript packet correctness, and UI/provider wiring.
QUICK START
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/rapidaai/voice-ai/blob/HEAD/.claude/skills/stt-integration/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/stt-integration, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
STT Integration Skill
Mission
Integrate STT providers that emit reliable interim/final transcripts, interruption signaling, and stt_latency_ms metrics.
Inputs expected from user
- Ingestion model: streaming WebSocket, SDK callback stream, or HTTP chunk/segment submit.
- Interim transcript support required or final-only acceptable.
- Required language/model controls.
If user does not answer:
- Follow nearest streaming baseline and preserve current transcript packet semantics.
Hard boundaries
In scope:
api/assistant-api/internal/transformer/<provider>/stt.go(+ provider option/callback helpers)api/assistant-api/internal/transformer/transformer.go- optional contract updates:
api/assistant-api/internal/type/stt_transformer.go,packet.go - provider STT JSON + UI config files/components
Out of scope:
- TTS-only transport changes
- telephony channel implementation
- EOS/VAD algorithm changes
Transport mapping
- WS callback baseline:
deepgram,assembly-ai,sarvam - SDK baseline:
azure,google,aws - HTTP/other baseline: provider-specific fallback keeping packet contract
Packet contract
Input:
UserAudioPacket
Required outputs:
SpeechToTextPacket(interim/final)InterruptionPacket{Source:"word"}when provider exposes speech-onset word signalConversationEventPacket{Name:"stt", ...}MessageMetricPacket{Name:"stt_latency_ms"}per utterance
Implementation workflow
- Select baseline provider by transport.
- Implement STT in provider folder and callback/parser helpers.
- Register provider in STT factory switch.
- Wire UI model/language metadata and provider form.
- Validate partial/final ordering and context IDs.
- Add tests for init, interim/final, reconnect/retry, latency metric.
Validation commands
go test ./api/assistant-api/internal/transformer/... -run TestSTTgo test ./api/assistant-api/internal/transformer/<provider>/...cd ui && yarn test providers./.claude/skills/stt-integration/scripts/validate.sh --check-diff --provider <provider>