What is SoulX-Podcast
SoulX-Podcast is a multi-speakertext-to-speech(TTS) model launched by Soul AI Lab, specifically designed for generating long-form podcast dialogues. The model has 1.7B parameters and supports Mandarin, English, and multiple Chinese dialects (such as Sichuanese, Henanese, Cantonese, etc.). It features cross-dialect prompting, enabling the generation of target dialect speech from Mandarin prompts. The model supports paralinguistic control (such as laughter, sighing, etc.), enhancing the realism of speech synthesis. SoulX-Podcast can generate coherent dialogues exceeding 90 minutes, maintaining stable timbre and emotional continuity, making it suitable for scenarios such as podcasts and audiobooks.
Main Features of SoulX-Podcast
- Multi-Speaker Support: Supports dialogue generation between multiple speakers, naturally switching between different speakers' voices, suitable for scenarios such as podcasts and audiobooks.
- Multilingual and Dialect Support: Supports Mandarin, English, and multiple Chinese dialects (such as Sichuanese, Henanese, Cantonese, etc.), features cross-dialect prompting, enabling the generation of target dialect speech from Mandarin prompts.
- Paralinguistic Control: Supports non-linguistic information (such as laughter, sighing, throat clearing, etc.), enhancing the realism of speech synthesis, making the generated speech more natural and vivid.
- Long-Form Dialogue Generation: Can generate coherent dialogues exceeding 90 minutes, maintaining stable timbre and emotional continuity, suitable for generating long-form podcast content.
- Zero-Shot Voice Cloning: Supports zero-shot voice cloning, capable of generating high-quality personalized speech without target speaker voice samples.
Technical Principles of SoulX-Podcast
- Base Model Architecture: Based on the Qwen3-1.7B architecture, a powerful pre-trained language model fine-tuned for multi-speaker dialogue generation tasks.
- Multi-Speaker Modeling: By introducing Speaker Embedding technology, the model can distinguish different speakers' voice characteristics and naturally switch between speakers during generation.
- Cross-Dialect Generation: Using the Dialect-Guided Prompting (DGP) method, the model can generate speech in the target dialect based on Mandarin prompts, supporting zero-shot generation of multiple dialects.
- Paralinguistic Control: By adding specific paralinguistic markers to the text input (such as
<|laughter|>、<|sigh|>, etc.), the model can incorporate corresponding non-linguistic information into the generated speech, enhancing the realism of the speech. - Long-Form Generation Stability: By optimizing the model's attention mechanism and decoder structure, it ensures stable timbre and emotional continuity in long-form dialogue generation, avoiding timbre drift and emotional incoherence issues.
- Data Processing and Training: Trained on large-scale multi-speaker dialogue data. The data processing pipeline includes speech enhancement, audio segmentation, speaker diarization, text transcription, and quality filtering, ensuring the model can learn rich dialogue features.
SoulX-Podcast Project Address
- Project Official Website：https://soul-ailab.github.io/soulx-podcast/
- GitHub Repository：https://github.com/Soul-AILab/SoulX-Podcast
- HuggingFace Model Library：https://huggingface.co/collections/Soul-AILab/soulx-podcast
- arXiv Technical Paper：https://arxiv.org/pdf/2510.23541
Application Scenarios of SoulX-Podcast
- Podcast Production: The model can generate coherent dialogue exceeding 90 minutes, suitable for producing podcasts on technology, culture, entertainment, and various other topics.
- Audiobooks: The model can generate dialogue among multiple characters, making audiobooks more vivid and engaging, suitable for long-form content such as novels and stories.
- Educational content: Generate multi-character dialogue to enhance the interactivity and engagement of educational content such as language learning and historical story narration.
- Entertainment and gaming: Generate natural multi-character speech for games, animation, and video production, enhancing the immersive feel of the content.
- Corporate training: Generate simulated dialogues to help employees with communication skills and customer service training.