Audio, transcription, and Realtime API

Beginner: choose by transport

Text-to-speech uses POST /v1/audio/speech and normally returns MP3, WAV, or other audio bytes. File transcription and translation use multipart requests to POST /v1/audio/transcriptions and /v1/audio/translations. Bidirectional realtime audio uses the GET /v1/realtime WebSocket. These are binary HTTP, form HTTP, and a full-duplex connection; they cannot share one JSON or SSE parser.

Create a site API key in token management, then confirm that each model supports the required audio endpoint in the model marketplace. The two values below are representative models, not a complete catalog:

Copyexport BASE_URL="https://api.tu-zi.com"
export API_KEY="your site API key"
export TTS_MODEL_NAME="gpt-4o-mini-tts"
export TRANSCRIBE_MODEL_NAME="gpt-4o-mini-transcribe"

BASE_URL has no trailing /v1. If the current key group cannot see either model, copy another exact model ID for the same endpoint from the signed-in marketplace.

Minimal TTS and transcription requests

Copycurl --fail --show-error "$BASE_URL/v1/audio/speech" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d @- \
  --output speech.mp3 <<JSON
{"model":"$TTS_MODEL_NAME","voice":"alloy","input":"Welcome to the audio API","response_format":"mp3"}
JSON

curl "$BASE_URL/v1/audio/transcriptions" \
  -H "Authorization: Bearer $API_KEY" \
  -F "model=$TRANSCRIBE_MODEL_NAME" \
  -F "file=@meeting.wav" \
  -F "response_format=json"

--fail makes an HTTP error fail instead of treating a common JSON error body as valid audio. Production code should also inspect status and Content-Type. Validate upload format, byte length, duration, and channels. When splitting long recordings, preserve time offsets and reconcile overlapping segments.

Realtime WebSocket lifecycle

A trusted server-side WebSocket client can connect to wss://api.tu-zi.com/v1/realtime?model=EXACT_MODEL_ID with Authorization: Bearer <site API key>. This site also accepts Sec-WebSocket-Protocol: realtime, openai-insecure-api-key.<site API key>. Configure a session, append audio, request a response, and consume events according to the selected model and channel contract.

Never embed a long-lived key in browser code. This site currently exposes no Realtime short-lived credential issuer, so a browser should connect through your controlled backend proxy instead.

Copyconnect → session configured → audio append → response create
        ← transcript/audio deltas ← response done or error

Bound session duration, idle time, message size, outbound queue, and heartbeat. If a consumer falls behind, drop reconstructible visualization frames or terminate the session rather than allowing an unbounded memory queue.

Errors, disconnects, and duplicates

A 400 on file endpoints usually indicates format, duration, or field problems; 413 means the upload is too large. Realtime 401/403 occurs at the handshake, while later failures arrive as events. Record session ID, response ID, event sequence, and Request-ID. After disconnect, create a new session by default. Replay only from a server-confirmed checkpoint when the protocol explicitly supports recovery, or you can duplicate transcription, speech, or tool effects.

Track the playback cursor once audio reaches the user. On interruption, stop local playback, clear pending audio, and send a supported cancellation event. Obtain lawful consent for audio capture and explain what is recorded and processed.

Production quality, privacy, and cost

Evaluate accents, noise, overlapping speakers, domain terms, numbers, language switching, time to first audio, and interruption. Do not log raw audio by default. Keep minimal duration, model, state, and trace metadata; encrypt retained recordings, separate access, and enforce deletion. TTS, transcription, and Realtime can use different billing units, so reconcile the model marketplace and usage logs instead of assuming a single per-minute rule.

Expert: latency budget and capacity protection

Split latency into capture, network, VAD, upstream first byte, synthesis, jitter buffer, and playback SLOs. Use bounded buffers, streaming I/O, and per-connection limits. A slow consumer must receive backpressure or disconnection. Load-test concurrent WebSockets, audio frames per second, codec CPU, egress bandwidth, and reconnect storms, and ensure proxies neither buffer WebSocket traffic nor let one connection exhaust an instance.