Replies in just over half a second: ElevenLabs launches Eleven Realtime voice models
ElevenLabs has introduced Eleven Realtime Mini and Max, two speech-to-speech models for voice agents. They support more than 90 languages and are compatible with OpenAI's Realtime interface.
Photo: Rafał Masłow / Wikimedia Commons, CC BY-SA 4.0 (cropped and resized)
Video: YouTube · ElevenLabs · youtube.com
AI audio company ElevenLabs has introduced Eleven Realtime, a new family of speech-to-speech models for voice agents. They come in two versions: Mini for assistants, tutors and interactive characters, and Max for customer service and multi-step workflows with tool calls. Time to first audio averages 0.57 seconds for Max and 0.62 seconds for Mini. More than 90 languages and the 17,000-plus voices in the ElevenLabs library are supported.
In tests by Artificial Analysis, Max scored 98.8 per cent on spoken reasoning (Big Bench Audio) and Mini 75.9 per cent; on tool-use tasks the scores were 87.1 and 60.3 per cent respectively. On natural conversational flow with pauses and interruptions, Mini was slightly ahead at 95.8 per cent. "You shouldn't have to become a configuration expert to build a voice agent that sounds natural," ElevenLabs wrote.
The models run on the ElevenAgents platform and follow the protocol of OpenAI's Realtime API, so developers can switch by changing the endpoint and model name. However, connections run over WebSockets only, so WebRTC apps need to be adapted. According to The New Stack, only Max offers a zero-retention mode.