On October 2, OpenAI announced the public testing of its real-time API designed for building AI applications that enable voice-to-voice interactions using GPT-4o. This new feature allows paid developers to create low-latency, multimodal interactive experiences within their applications.
OpenAI also revealed partnerships with three voice API collaborators: LiveKit, Agora, and Twilio. Agora, which focuses on the U.S. and international markets, has released a conversational AI SDK that integrates OpenAI's new real-time API, facilitating natural voice interactions with AI.
This approach processes voice directly rather than converting it to text, enabling realistic conversations and allowing AI to comprehend human emotions. The launch of the real-time API marks a significant advancement for OpenAI in the AI application space, reducing interaction delays and enhancing emotional expression in conversations.