NovaSynth is a pre-production testing and validation tool for AI agents, with the core goal of letting development teams "break" their own agents before real customers encounter problems. It sends synthetic users to the agent under test through real telephone networks, real-time audio, and chat, creating various kinds of real-world chaos, thereby exposing failures during rehearsal and validating fixes end to end.
Purpose
NovaSynth is used for pre-release stress testing and reliability validation of AI voice and chat agents. It advances testing from "a small number of ideal conversations" to a rehearsal of "real-world chaos," helping teams evaluate complete multi-turn conversation journeys rather than the quality of a single reply. Its typical goal is to discover the points where agents fail under the noise, emotion, interruptions, and adversarial intent brought by real callers.
Main capabilities
- Synthetic users and real chaos: Synthetic users carry goals, emotions, accents, and behavioral characteristics, and can simulate frustrated repeat callers, irritable users stuck in traffic, users who switch languages mid-sentence, and more. Specific examples given in the materials include: calling for the third time because of a delayed refund, canceling midway and changing their mind, fast interjectors who interrupt and talk over others, and prompt injection attempts that ask the agent to output system prompts and tool definitions.
- Real telephony and audio: Through SIP dialing via Twilio, Plivo, or Vobiz, as well as LiveKit real-time audio, it provides real phone calls, real audio, and real latency, rather than simulated conversation transcripts.
- Persona and scenario generation: Systematically generates personas with goals, emotions, and accents, as well as corresponding scenarios, such as frustrated repeat callers, non-native speakers, and prompt injection probes, rather than writing them by hand.
- Batch matrix runs: Runs a complete matrix of persona x scenario under a shared batch ID, covering the combinatorial space and grouping comparisons in the same place.
- Concurrency and load testing: Increasing concurrency shows how many calls the agent can handle simultaneously, and load testing supports up to 2,000 concurrent lines; ordinary functional testing remains at low concurrency, and advance notice is required before large-scale load testing so lines can be prepared.
- Tool virtualization: Read operations are replayed, and write operations are sandboxed, so that changed decisions are exercised end to end while write operations do not reach production systems.
- Capture and rescoring: Every synthetic call is captured as a complete trace and scored using 30+ voice AI metrics (100+ across the platform), including the audio itself, so that failures emerge during rehearsal.
- Queue and scaling architecture: Asynchronous queue scheduling and parallelized execution based on BullMQ, PostgreSQL, and Redis,Each run can support hundreds of scenarios without blocking the test environment.
How to Use
Teams point NovaSynth at an agent under test—whether it is a test number, a staging SIP, or a new agent that has not yet been released. For single-agent architectures, NovaSynth can be placed into the Agent configuration, while also providing the PRD, BRD, workflow documents, or any context describing the agent's responsibilities; the fuller the context, the more precise the generated personas, scenarios, and suggestions. For multi-agent architectures, it is not recommended to paste a single prompt; instead, integrate its SDK so the system can obtain all information from traces. Descriptive context alone can also be provided (for example, loan approval, debt collection, customer support), and simulations, evaluations, and reports can still be obtained, but without visibility into the agent's decision-making process, suggestions will be less precise. The platform can connect to telephony SIP (Twilio, Plivo, Vobiz), LiveKit, Pipecat, LangChain/LangGraph, OpenAI and Anthropic SDKs, and HTTP-based chat; custom or from-scratch agents can be integrated through the SDK or lightweight hooks around LLM calls. Users can also submit their own scenarios and observe the system run, and the system will also suggest use cases that may have been missed based on the context.
Applicable Scenarios
Suitable for teams that need to validate multi-turn voice or chat journeys before release, especially in scenarios such as customer support, refund processing, billing collection, and loan approval. The materials explicitly mention extending from testing the ideal path to rehearsing real-world chaos, and covering situations such as interruptions, talking over each other, silence, topic switching, accents, and language switching.
Limitations and Boundaries
NovaSynth is positioned as a pre-production module and will not simulate callers to evaluate the production environment; production evaluation belongs to a separate subsequent module. Although load testing supports up to 2,000 concurrent sessions, advance notice is required before large-scale load runs to prepare lines. It can still run without agent context, but the precision of suggestions will decrease. Regarding charges, pricing, and specific deployment environment requirements, the materials do not currently provide them.
