OpenParable
Skip to content
← Back to skills

Stt Integration

Development
stt-integration

Stt Integration is an Agent Skill for development published in the rapidaai/voice-ai repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Add or modify speech-to-text providers in assistant-api with transport-aware ingestion (WS/SDK/HTTP), transcript packet correctness, and UI/provider wiring.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/rapidaai/voice-ai/blob/HEAD/.claude/skills/stt-integration/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/stt-integration, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

STT Integration Skill

Mission

Integrate STT providers that emit reliable interim/final transcripts, interruption signaling, and stt_latency_ms metrics.

Inputs expected from user

  1. Ingestion model: streaming WebSocket, SDK callback stream, or HTTP chunk/segment submit.
  2. Interim transcript support required or final-only acceptable.
  3. Required language/model controls.

If user does not answer:

  • Follow nearest streaming baseline and preserve current transcript packet semantics.

Hard boundaries

In scope:

  • api/assistant-api/internal/transformer/<provider>/stt.go (+ provider option/callback helpers)
  • api/assistant-api/internal/transformer/transformer.go
  • optional contract updates: api/assistant-api/internal/type/stt_transformer.go, packet.go
  • provider STT JSON + UI config files/components

Out of scope:

  • TTS-only transport changes
  • telephony channel implementation
  • EOS/VAD algorithm changes

Transport mapping

  • WS callback baseline: deepgram, assembly-ai, sarvam
  • SDK baseline: azure, google, aws
  • HTTP/other baseline: provider-specific fallback keeping packet contract

Packet contract

Input:

  • UserAudioPacket

Required outputs:

  • SpeechToTextPacket (interim/final)
  • InterruptionPacket{Source:"word"} when provider exposes speech-onset word signal
  • ConversationEventPacket{Name:"stt", ...}
  • MessageMetricPacket{Name:"stt_latency_ms"} per utterance

Implementation workflow

  1. Select baseline provider by transport.
  2. Implement STT in provider folder and callback/parser helpers.
  3. Register provider in STT factory switch.
  4. Wire UI model/language metadata and provider form.
  5. Validate partial/final ordering and context IDs.
  6. Add tests for init, interim/final, reconnect/retry, latency metric.

Validation commands

  • go test ./api/assistant-api/internal/transformer/... -run TestSTT
  • go test ./api/assistant-api/internal/transformer/<provider>/...
  • cd ui && yarn test providers
  • ./.claude/skills/stt-integration/scripts/validate.sh --check-diff --provider <provider>