Roark is a QA and evaluation (evals) platform for voice AI, positioned to test, monitor, and improve the voice AI agents you already run, rather than being another agent builder. It is built around a continuous improvement loop: find problems in production, verify fixes in simulation, launch with evidence, and then Roark keeps watching the next call.
Pre-launch simulation testing Before customers call in, put the agent through drills against hundreds of simulated callers. These simulated callers come from your real call types and include realistic personas (angry callers, ramblers, interrupters, etc.), accents, background noise, and edge cases. Roark also provides adversarial callers that attempt prompt injection, jailbreaks, and social engineering to test whether the agent follows policy under attack. Simulation covers 45 languages and accents, including native accents, code-switching, and background noise. It supports peak concurrent load testing and continuous health checks to ensure that an agent that passes in pre-production can survive launch day. Regression testing can rerun the entire suite after every change and diff against the last green baseline, so you do not fix one caller and break another. Every prompt or model change runs the test suite before merge, effectively setting a quality gate for conversations.
Post-call analytics Every production call is scored on ingestion: problems are automatically triaged, alerts are triggered, and dashboards and OTEL tracing are provided, for both voice calls and chat threads. Unlike most tools that use an LLM to score transcripts, Roark runs purpose-built audio models on the call itself, measuring what the customer actually heard. The platform provides 500+ audio-native metrics covering dimensions such as audio-native, conversational, compliance, and latency, for example script adherence, first response time, interruption handling, silence duration, and pronunciation testing (drug names, SKUs, and person names verified from the waveform). Custom metrics are supported to automatically score every call according to your rubric. Human review and ground-truth labeling are also available to label real calls, set the correct answers, and measure how well each metric agrees with human judgment. An issue tracker triages failures and clusters them, tracking them across deployments. Alerts and monitoring can set thresholds on any metric and push to Slack or a webhook. Tracing and dashboards provide OTEL spans for each conversation turn, as well as trend scores by agent and version.
Continuous improvement Roark provides a prompt optimizer that drafts evidence-backed prompt changes from failed calls; ground-truth tuning can adjust metrics according to your labels until the model agrees with the team's judgment; and when the problem is not wording, it provides replacement and setup recommendations for models, voices, and infrastructure.; fix validation can replay your fix onto the exact callers that originally had the problem; post-launch validation observes metrics starting from the first call after deployment.
How to use Roark can be integrated with one click on any of the platforms below, and production calls will flow in automatically; you can also send any recording with just a small amount of code. It supports use with Vapi, Retell, LiveKit, Pipecat, and your own stack. Signing up lets you start for free, with a $50 credit included and no credit card required.
Use cases
- Customer service and sales calls: score the pronunciation, empathy, and resolution rate of every conversation.
- Medical calls: automatically compliance-score disclosure and identity verification for every conversation.
- Customer voice agents: validate through simulation before launch and use evidence to prove the build is ready.
Other information Roark also provides a public real-time voice AI leaderboard, Roark Live Bench, which places calls across the real telephone network for every combination of speech-to-text, language model, and voice engine, and scores them end to end with the same scoring engine. Each evaluation is published under a suite version, keeping it comparable with the previous one. It can be read for free without registration. The platform provides enterprise-grade capabilities from day one: annual penetration testing, SSO/SAML, role-based access control, and configurable data retention policies.
Limitations and information not provided The materials do not provide a specific pricing plan (other than the free credit), whether each feature is charged separately, a complete list of supported agent platforms, specific deployment methods, data storage locations, SLA, or other detailed information; the above content is not currently provided.
