OpenParable
Skip to content
← Back to skills

Agf Wiring Multi LLM SDK

Development
agf-wiring-multi-llm-sdk

Agf Wiring Multi LLM SDK is an Agent Skill for development published in the pcliangx/AppGenesisForge repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use when wiring up or switching between China-domestic LLM providers (DeepSeek, Doubao/Volc Ark, Qwen/DashScope, MiniMax). Provides OpenAI-compatible adapter pattern, env-var contracts, fallback strategy, cost guardrails, and minimum verifications before declaring integration done.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/pcliangx/AppGenesisForge/blob/HEAD/.claude/skills/agf-wiring-multi-llm-sdk/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/agf-wiring-multi-llm-sdk, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Wiring Multi-LLM SDK (DeepSeek / Doubao / Qwen / MiniMax)

Use this skill when:

  • You add a new LLM provider to backend/app/agents/ or any backend module
  • You switch the default provider (e.g. DeepSeek → Doubao for cost reasons)
  • You wire fallback / retry logic between providers
  • You suspect a provider mismatch is the cause of a bug

Decision: which SDK style?

All four providers expose OpenAI-compatible endpoints. Default to the openai Python SDK with a custom base_url rather than each vendor's bespoke SDK — fewer dependencies, easier to swap, less drift.

Bespoke SDK exceptions:

  • Doubao multimodal (image/video gen): use volcengine-python-sdk for Ark image API
  • MiniMax video / TTS: use minimax official SDK
  • Streaming nuance: confirm OAI-compat client handles vendor's stream chunk format

Before wiring any SDK, pull its current docs via Context7 (resolve-library-id → query-docs) — all four vendors iterate fast and training-data memory of their APIs is likely stale. Context7 coverage of domestic SDKs varies; if a library isn't indexed, fall back to WebFetch on official docs.

Env var contract (locked)

All providers follow the same pattern. Never hardcode keys. Each is read from environment at module init; a missing key raises early.

ProviderEndpoint envKey envDefault model env
DeepSeekDEEPSEEK_BASE_URL (default https://api.deepseek.com/v1)DEEPSEEK_API_KEYDEEPSEEK_MODEL (e.g. deepseek-chat)
Doubao (Volc Ark)ARK_BASE_URL (default https://ark.cn-beijing.volces.com/api/v3)ARK_API_KEYARK_MODEL_ENDPOINT_ID (vendor-specific endpoint id, NOT model name)
Qwen (DashScope)DASHSCOPE_BASE_URL (default https://dashscope.aliyuncs.com/compatible-mode/v1)DASHSCOPE_API_KEYQWEN_MODEL (e.g. qwen-plus)
MiniMaxMINIMAX_BASE_URL (default https://api.minimaxi.com/v1)MINIMAX_API_KEYMINIMAX_MODEL (e.g. abab6.5s-chat)

Doubao gotcha: the "model name" in OAI-compat call is actually the Ark endpoint id (ep-2024xxxx), not a public model id like doubao-pro-32k. Get the endpoint id from Volc Ark console.

Adapter skeleton (Python / FastAPI)

# backend/app/agents/llm_clients.py
import os
from openai import OpenAI

def get_client(provider: str) -> tuple[OpenAI, str]:
    if provider == "deepseek":
        return OpenAI(
            api_key=os.environ["DEEPSEEK_API_KEY"],
            base_url=os.getenv("DEEPSEEK_BASE_URL", "https://api.deepseek.com/v1"),
        ), os.getenv("DEEPSEEK_MODEL", "deepseek-chat")
    if provider == "doubao":
        return OpenAI(
            api_key=os.environ["ARK_API_KEY"],
            base_url=os.getenv("ARK_BASE_URL", "https://ark.cn-beijing.volces.com/api/v3"),
        ), os.environ["ARK_MODEL_ENDPOINT_ID"]
    if provider == "qwen":
        return OpenAI(
            api_key=os.environ["DASHSCOPE_API_KEY"],
            base_url=os.getenv("DASHSCOPE_BASE_URL", "https://dashscope.aliyuncs.com/compatible-mode/v1"),
        ), os.getenv("QWEN_MODEL", "qwen-plus")
    if provider == "minimax":
        return OpenAI(
            api_key=os.environ["MINIMAX_API_KEY"],
            base_url=os.getenv("MINIMAX_BASE_URL", "https://api.minimaxi.com/v1"),
        ), os.getenv("MINIMAX_MODEL", "abab6.5s-chat")
    raise ValueError(f"unknown provider: {provider}")

Fallback strategy

Default order (tunable in CLAUDE.md per project): DeepSeek → Doubao → Qwen → MiniMax.

  • 5xx / network → next provider
  • 4xx (auth / quota) → DO NOT failover; raise (signals config bug)
  • Latency P95 breach (> 5s for non-streaming) → tier down silently, log warning

Implement with tenacity retry + a thin orchestrator that walks the list. Never silently swap models without telemetry — every failover emits a structured log line per observability.md.

Token + cost telemetry

Every call must record the LLM fields mandated by observability.md:17 plus provider. DeepSeek + Doubao support prompt caching — read cache_hit_ratio off the response usage object.

Verifications before "done"

Before claiming the integration works, run this checklist explicitly. Verify outputs match expectations — do not assume.

  • Smoke: each enabled provider answers "你好" with non-empty text and >0 token usage
  • Streaming: chunk delivery is real (not buffered all-at-once); first-token latency logged
  • Failover: simulate one provider down (point base_url at localhost:1) and confirm auto-fallback + log line
  • Cost: token usage written to log + DB on every call
  • Secrets: no key appears in logs / git diff (run git diff | grep -iE 'api[_-]?key|secret|token' before commit)
  • Unit test mocks at the OpenAI SDK boundary (not at HTTP level) — survives base_url changes
  • SDK surface: API calls cross-checked against current docs (Context7 or official site) — not from training memory

Anti-patterns

  • ❌ Hardcoding base_url strings — always read from env
  • ❌ Passing public model id to Doubao — use endpoint id
  • ❌ Catching all exceptions and returning empty string — masks auth failures and quota issues
  • ❌ Assuming all providers return the same usage shape — validate and normalize
  • ❌ Storing keys in CLAUDE.md / settings.json — only .env (gitignored) or secret manager

References