openai-realtime-audio

openai-realtime-audio

OpenAI Realtime API — covering low-latency speech-to-speech sessions over WebRTC/WebSocket/SIP, and the Audio endpoints (file transcription, text-to-speech).

Directory Structure

skills/openai-realtime-audio/
  SKILL.md
  references/
    realtime/
      README.md
      overview.md
      webrtc.md
      websocket.md
      sip.md
      conversations.md
      vad.md
      transcription.md
      translation.md
      tools-mcp.md
      server-controls.md
      costs.md
      voice-agents.md
      models-prompting.md
      client-secrets.md
    audio/
      README.md
      audio-and-speech.md
      file-transcription.md
      text-to-speech.md
      transcription-workflow.md
  samples/
    README.md
    webrtc-browser-session.md
    websocket-server-connection.md
    client-secret-issuance.md
    realtime-transcription-stream.md
    file-transcription.md
    text-to-speech-synthesis.md
    voice-agent-basic.md

scripts/ is intentionally not created. This is because the connection code (WebRTC/WebSocket setup, client secret issuance, etc.) is fully covered on the samples/ side.

Exploration Procedure

Derive the category from the task, and identify the target page in the category's README.md:

1. Find the category corresponding to the task in the mapping table below
2. of that category references/{category}/README.md（samples/ (the README.md directly under it) to identify the target page
3. Read the .md of the relevant page to confirm the details

For general webhook and Responses API references, see the openai-api-core skill. For Agents SDK-side voice agents (RealtimeAgent / RealtimeSession and other SDK wrappers), see the openai-agents skill. This skill handles the Realtime API itself (WebRTC/WebSocket/SIP transport, session/event specs, client secrets) and the Audio endpoints (file transcription, text-to-speech). Realtime The keyword is unrelated to Supabase's Realtime (a database pub/sub product), so do not confuse them.

Task → Category Mapping

Task	Category	Reference README
Want to establish a Realtime session over WebRTC/WebSocket/SIP	realtime	references/realtime/README.md
Want to know about conversation/event lifecycle, VAD, tools/MCP	realtime	references/realtime/README.md
Want to know about client secret (ephemeral key) issuance, cost, and prompting	realtime	references/realtime/README.md
Want to know about file audio transcription and text-to-speech (non-realtime)	audio	references/audio/README.md
Want to know how to choose a transcription workflow (file vs realtime)	audio	references/audio/README.md
Want to know typical usage	samples	samples/README.md