SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI audioCekura Bench

Cekura Bench

The Speech-to-Speech benchmark page of Cekura Bench compares and evaluates the performance of multiple speech-to-speech models in real WebRTC real-time audio calls for real-time voice agent scenarios, covering metrics such as reliability (success rate), data accuracy, silence/no-response rate, response time, and cost per minute. The evaluation includes two types of call scenarios, Booking and Medicare, and publishes each model's pass rate by scenario. Models generally perform well in Booking calls, while in the more demanding Medicare calls, only a few models pass more than half. The page emphasizes that no model is superior across all dimensions, and users need to choose based on the metrics they care about most. The data was last updated on October 7, 2026.

Open website

About this tool

Cekura Bench provides benchmark evaluations of speech-to-speech (S2S) models, focusing on the performance of real-time voice agents in real calls. The evaluation dimensions include reliability (task success rate), data accuracy, response time, the proportion of going quiet, and cost per minute, and it has each AI voice model handle incoming calls in a real WebRTC real-time audio call environment to test its actual performance.

  1. Evaluation Dimensions and Ranking Basis The page uses "success rate" (pass³, i.e., the proportion of scenarios that passed in all three runs) as the core ranking basis, while also displaying data accuracy, silence rate, response time, and cost per minute. The evaluation notes state that higher and further to the left (i.e., high reliability and short response time) is better. The individual leaders are: the highest reliability is GPT Realtime 2.1 (79.3% pass³), the fastest reply is Phonic v1 (median response time 1.60 seconds), and the lowest cost is GPT Realtime 2.1 Mini ($0.020 per minute).

  2. Models Participating in the Evaluation and Key Data Ranked by pass³ from high to low:

  3. GPT Realtime 2.1 (OpenAI): success rate 79.3%, data accuracy 89.0%, silence 3.3%, response time 1.94s, cost $0.085/minute. Settings: model gpt-realtime-2.1, reasoning high, voice marin, turn management Service turn detection. Comment: almost never makes mistakes, performs best in Medicare calls, but has difficulty with noisy audio, occasionally makes errors saving phone numbers or forgets previous content.

  4. GPT-Live 1 + GPT-6 Sol (OpenAI): success rate 75.6%, data accuracy 83.7%, silence 12.2%, response time 2.02s, cost $0.055/minute. Settings: gpt-live-1, Sol on low thinking, voice marin. Comment: Booking calls are nearly perfect, but in Medicare calls it transfers to the wrong team or skips steps, has the highest silence rate, and the lowest voice rating.

  5. Azure Realtime (Microsoft): success rate 70.7%, data accuracy 84.1%, silence 4.1%, response time 1.66s, cost $0.080/min. Settings: azure-realtime, no reasoning setting, voice ava. Comment: Best at handling caller self-corrections, passes all interrupted calls and tool-call-failure calls, response speed among the fastest; but often misses ID in Medicare calls, prone to errors in background noise and multi-request calls.

  6. Phonic v1 (Phonic): success rate 68.3%, data accuracy 77.2%, silence 0.0%, response time 1.60s, cost $0.140/min. Settings: phonic_v1, Intelligence high, voice sabrina. Comment: Fastest responses and never keeps callers waiting, one of the best for booking calls, but fails strict Medicare checks and makes too many tool calls.

  7. Grok Voice Think Fast 2.0 (xAI): success rate 67.1%, data accuracy 76.0%, silence 4.5%, response time 1.62s, cost $0.080/min. Settings: grok-voice-think-fast-2.0, reasoning high, voice eve. Comment: Shortest replies, stable booking; loses or scrambles early details in Medicare calls, silences all occur in Medicare calls.

  8. Gemini 3.1 Flash Live Preview (Google): success rate 62.2%, data accuracy 76.0%, silence 3.3%, response time 2.88s, cost $0.070/min. Settings: models/gemini-3.1-flash-live-preview, Thinking level high, voice Charon. Comment: Reliable for booking but slowest responses, replies lag or go missing after multiple tool calls, Medicare transfer information incorrect.

  9. Deepslate Opal (Deepslate): success rate 57.3%, data accuracy 69.9%, silence 5.3%, response time 2.27s, cost $0.112/min. Settings: opal, no reasoning setting, voice ElevenLabs 21m00Tcm4TlvDq8ikWAM. Comment: Passes all interrupted and multi-request calls, but often skips steps in Medicare calls, weakest against silence, coughing, and packet loss.

  10. Gemini 3.8 Live Extended Thinking (Google): success rate 52.4%, data accuracy 76.0%, silence 5.3%, response time 2.07s, cost n/a. Settings: gemini-3.8-live-extended-thinking, Thinking level high, voice Charon. Comment: One of the strongest models for Medicare calls, but weakest for ordinary booking, unpredictable between runs, goes silent under background conversation.

  11. Amazon Nova 2 Sonic (Amazon): success rate 50.0%, data accuracy 59.3%, silence 1.2%, response time 2.18s, cost n/a. Settings: amazon.nova-2-sonic-v1:0, Endpointing medium, voice matthew. Comment: Best voice rating, but misses names and phone numbers, rarely passes Medicare, most repeated tool calls and longest replies.

  12. GPT Realtime 2.1 Mini (OpenAI): success rate 50.0%, data accuracy 61.4%, silence 2.5%, response time 2.08s, cost $0.020/min. Settings: gpt-realtime-2.1-mini, reasoning high, voice marin. Comment: Completes most bookings at extremely low cost, but forgets early details, not a single Medicare record accurate, mishears phone numbers and struggles to recover from tool errors.

  13. Gemini 3.8 Live (Google): success rate 41.5%, data accuracy 69.1%, silence 2.0%, response time 1.91s, cost n/a. Settings: gemini-3.8-live, no thinking level, voice Charon. Comment: Faster responses and more bookings completed, but more Medicare record errors, worst at handling caller self-corrections, most unpredictable. Also a reference item (not ranked): Cascade (Cascade), success rate 82.9%, data accuracy 93.1%, silence 0.8%, response time 2.17s, cost n/a. Its architecture is speech-to-text → LLM → text-to-speech (flux-general-en → gpt-4.1 → eleven_flash_v2_5). Comment: Passes every time in most scenarios, almost never keeps callers waiting, but the replies are not the fastest, and during Medicare calls it occasionally forgets earlier content.

III. Results by call type The evaluation is divided into booking calls and Medicare calls. Booking calls are simpler, and most models get them right almost every time; Medicare calls are far more demanding, and only a few models pass more than half. The page lists each model's success rates for the two call types and the gap between them, for example: GPT Realtime 2.1 booking 86.4%, Medicare 60.9% (gap 25.6 percentage points); Cascade booking 93.2%, Medicare 56.5% (gap 36.7); Azure Realtime booking 81.4%, Medicare 43.5% (gap 37.9); GPT-Live 1 + GPT-6 Sol booking 91.5%, Medicare 34.8% (gap 56.7); Gemini 3.8 Live Extended Thinking booking 59.3%, Medicare 34.8% (gap 24.5); Deepslate Opal booking 72.9%, Medicare 17.4% (gap 55.5); Gemini 3.8 Live booking 52.5%, Medicare 13.0% (gap 39.5); Gemini 3.1 Flash Live Preview booking 83.1%, Medicare 8.7% (gap 74.4); Grok Voice Think Fast 2.0 booking 91.5%, Medicare 4.3% (gap 87.2); Amazon Nova 2 Sonic booking 67.8%, Medicare 4.3% (gap 63.4); GPT Realtime 2.1 Mini booking 67.8%, Medicare 4.3% (gap 63.4); Phonic v1 booking 93.2%, Medicare 4.3% (gap 88.9).

IV. Failure modes and selection advice The page points out that two models may score the same but fail for different reasons: some make mistakes on details, some go silent, and some pass on one run but fail on another. The page explicitly states that "no model wins on all fronts": callers want correct answers and fast replies, users want low bills, and there is no singlemodel that can satisfy all three at once, so you need to choose based on the metric you care about most. The evaluation data was last updated on October 7, 2026, and the methodology was provided by Dileep Chagam. Regarding the cost of using this benchmark page, how the platform is opened up, detailed methodology steps, and similar information, the materials do not currently provide it.

Related tools

Browse category