Voice Engine - OpenAI's AI Voice Synthesis and Voice Cloning Model

What is Voice Engine

Voice Engine is a newly announced AI voice synthesis and voice cloning technology from OpenAI that can generate natural-sounding speech close to the original voice using a short 15-second audio sample and text input. Since its development in late 2022, this technology has been applied in OpenAI's text-to-speech API and ChatGPT's voice features. Voice Engine has broad application prospects, including providing reading assistance for children and non-readers, translating content to reach global audiences, supporting non-verbal communicators, and helping restore patients' voices. At the same time, to ensure the safe use of the technology, OpenAI has established strict usage policies to prevent voice impersonation and has taken multiple safety measures, including watermark tracking.

Official blog introduction:https://openai.com/blog/navigating-the-challenges-and-opportunities-of-synthetic-voices

Voice Engine's application scenarios and cases

• Education and reading assistance: Voice Engine can provide natural-sounding speech for children and non-readers, helping them better access and understand written content. For example, the educational technology company Age of Learning uses this technology to generate voice content for preset scripts and combines it with GPT-4 to create real-time, personalized interactive responses to improve students' learning experience.
• Content translation and globalization: Through Voice Engine, content such as videos and podcasts can be translated into the listener's native language while preserving the original speaker's accent, enabling creators and businesses to reach global audiences in a more authentic and approachable way. For example, the AI visual storytelling platform HeyGen uses Voice Engine for video translation, allowing its content to cross language barriers and reach a broader audience.
• Improving service delivery in remote areas: Voice Engine can improve basic services in remote communities, such as health consultations, by providing services in local languages. Dimagi is developing tools that use Voice Engine and GPT-4 to provide interactive feedback to community health workers, helping them improve their skills.
• Supporting people with speech disabilities: For individuals with communication difficulties, Voice Engine can provide unique and non-robotic voices, enabling them to communicate through augmentative and alternative communication (AAC) devices. Livox uses this technology to provide its users with natural-sounding voices in multiple languages, allowing them to express themselves better.
• Restoring patients' voices: For patients who have lost the ability to speak due to illness or neurological problems, Voice Engine can help them restore or rebuild their voices. For example, the Norman Prince Neurosciences Institute is exploring how to use Voice Engine to help individuals with speech impairments caused by tumors or neurological reasons.

Voice Engine audio examples

1. Audio translation

Reference English audio

Generated Chinese audio

2. Patient voice restoration

Patient's original voice

Reference audio

Generated audio

How to use Voice Engine

Voice Engine is currently still in a small-scale preview stage and has not been widely released. OpenAI is conducting private testing with a small group of trusted partners to better understand the technology's possible uses, and based on the results of these small-scale tests and the feedback received, it will decide how and whether to deploy this technology more broadly in the future.

Therefore, Voice Engine is currently not open for public online use. OpenAI has taken a cautious approach to ensure that while promoting this technology, it can fully consider its potential misuse risks and formulate corresponding safety measures and usage policies. In the future, OpenAI may decide whether to make Voice Engine available to a broader user base based on test results and the needs of social development.