Speech Generation

The self-hosted gateway serves the OpenAI Speech API beside the chat, image, and video endpoints: POST /v1/audio/speech takes a JSON request and answers with binary audio, forwarded to the caller as it arrives. Which models answer is your configuration: a model must declare audio output (output_modalities), and its candidates bill per million counted characters. GET /v1/models lists every model's output_modalities, so clients can discover the speech models an instance serves.

Quick start

curl http://localhost:8080/v1/audio/speech \
  -H "Authorization: Bearer sk-yr-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"cosyvoice","input":"你好，欢迎来到自托管网关。","voice":"your-voice-id"}' \
  -o welcome.mp3

The official OpenAI SDK works unchanged — point base_url at the instance and call client.audio.speech.create.

What the gateway reads

model, input, and voice are required; response_format (one of mp3, opus, aac, flac, wav, pcm) and speed are optional. Unknown fields are ignored, so newer SDKs do not break older gateways. Two OpenAI parameters are refused with a clear 400 rather than silently dropped: instructions (none of the wired dialects serves it) and stream_format (only whole-byte streaming exists here — which is the default, and needs no flag).

voice passes through verbatim: voice vocabularies are each vendor's own, per model. An unknown voice is the upstream's 400 to make, and the caller sees the status. Input length limits are the upstream's too — MiniMax's synchronous endpoint accepts 10,000 characters, Zhipu's 1,024, and an over-limit request is answered with the upstream's own error; SiliconFlow publishes no number — live probing shows its synchronous endpoint starts refusing erratically with 5xx around ~7,000 ASCII characters (the same length flips between accepted and refused across retries — at the vendor's speaking pace roughly 80–90 seconds of audio), so split long texts yourself.

Speech dialects

Every base this build does not know by name is spoken to in the OpenAI speech shape itself. Three hosts carry their own dialect:

• SiliconFlow (api.siliconflow.cn): the OpenAI shape with mp3/opus/wav/pcm (mp3 the default). The vendor bills per UTF-8 byte of input, so the meter counts bytes — the bill follows the invoice, not a rune count. Preset voices are model-prefixed (FunAudioLLM/CosyVoice2-0.5B:alex; the system presets are eight — alex, benjamin, charles, david, anna, bella, claire, diana; a bare name or an omitted voice is the upstream's own 400).
• Zhipu (open.bigmodel.cn): wav and pcm only, and an unspecified caller gets wav — never a mislabeled default mp3. A format the dialect does not serve is refused before anything is dialled, with the served set named. Seven system voices: tongtong (the default), chuichui, xiaochen, and the four jam/kazi/ douji/luodo presets, plus cloned-voice ids from the vendor's voice-clone API. (Configure the provider base as https://open.bigmodel.cn/api/paas/v4 — the versioned root; a bare host would get /v1 appended and dial a path that does not exist.)
• MiniMax (api.minimax.cn): the t2a_v2 endpoint — formats mp3/pcm/wav/flac/opus (mp3 the default), voice and speed ride inside voice_setting, and the answer is one JSON envelope with hex-encoded audio, which the gateway decodes and forwards. The envelope's own usage_characters prices the bill when present (a stated count wins, however small); without it the gateway re-counts the text in the vendor's rule — one CJK glyph counts as two. A refusal inside the 200 (base_resp) is answered 422 with the vendor's code and message; non-2xx statuses keep the vendor's wording on the audit row and the caller sees the status.

Billing

A speech candidate carries one price — CNY per million counted characters — and the count is metered in the settling provider's own counting rule (the examples above: bytes at SiliconFlow, the vendor's character rule at MiniMax, plain characters elsewhere). Before anything is dialled, the cheapest enabled candidate's estimate is held against the key's budget ceiling (a certain refusal answers 429 and spends nothing upstream); settlement then bills the count the delivery actually reported — at MiniMax that means the invoice's own number, correcting the estimate if the two differ. Because the meter is the provider's own rule, the same model behind different providers' candidates may meter the same text differently — check the request log's usage detail, which names the meter each bill used. A request that produced no audio bills nothing; a stream cut after synthesis started still bills what was synthesised.

Voice identity: no cross-provider failover

The voice is a thing the caller named, and voices do not travel between vendors — so a failed speech request is answered as an error, never silently re-served by another provider speaking a different voice. The candidate walk stops at the first provider (key rotation inside that one provider keeps working: same provider, same voice). Chain a second provider's candidate for supply, but know it serves only when the head is skipped entire — a failure surfaces the head's own answer.