ByteByteGo14 min readintermediate
How OpenAI Built GPT-Live
Summary
OpenAI’s GPT‑Live‑1 is a full‑duplex voice model that can listen and speak simultaneously by tokenizing audio (including silence) and emitting tokens on an ~80 ms clock. By keeping the model small and fast and delegating heavy reasoning to a separate LLM, OpenAI solves the turn‑detector problem of earlier cascaded and turn‑based systems while meeting sub‑100 ms latency requirements. The post walk…
- Voice assistants have evolved from cascaded ASR → LLM → TTS pipelines, to end‑to‑end turn‑based speech‑to‑speech models, and now to full‑duplex models that process and generate audio continuously.
- Full‑duplex models treat silence as just another token, allowing a single model to decide when to listen, speak, or stay silent without a separate turn‑detector.
- Running a model that predicts every 80 ms frame is compute‑heavy; GPT‑Live‑1 keeps the model small enough to meet millisecond‑scale latency while offloading expensive reasoning to a separate, higher‑capacity LLM.
- Delegating “thinking” to an async path avoids long pauses in the conversation, enabling the voice front‑end to keep talking while the backend performs search, tool calls, or complex reasoning.
Real‑time conversational AI must feel natural; eliminating the turn‑detector removes awkward pauses and interruptions that have plagued voice assistants for years. GPT‑Live demonstrates a practical path to low‑latency, full‑duplex speech interaction, which is a prerequisite for next‑generation assi…
6/10
