What is SoulX-Singer
SoulX-Singer is an industrial-grade zero-shotsinging voice synthesismodel open-sourced by Soul App in collaboration with Tianjin University and Northwestern Polytechnical University. The model is trained on 42,000 hours of high-quality multilingual singing voice data, supports dual-mode control via MIDI scores and F0 melodies, and enables precise pitch and rhythm control, cross-lingual timbre cloning, and lyric editing. SoulX-Singer adopts an advanced Flow Matching architecture and a two-stage training strategy, comprehensively leading existing open-source solutions in key metrics such as pitch accuracy, singer similarity, and subjective listening perception, providing reliable infrastructure for AI music creation and virtual singer applications.
SoulX-Singer's Main Features
-
Zero-shot singing voice cloning: Input any singer's reference audio to generate high-quality singing in that timbre without additional training.
-
Dual-mode control synthesis: Precisely control pitch and rhythm via MIDI scores, or convert humming to singing via F0 melodies.
-
Multilingual singing voice synthesis: Supports high-quality singing voice generation in Mandarin, English, and Cantonese.
-
Cross-lingual timbre transfer: Transfer the timbre characteristics of a singer in one language to singing in other languages.
-
Real-time lyric editing: Flexibly modify lyric content while keeping the melody and singing style unchanged.
SoulX-Singer's Technical Principles
- Flow Matching generation framework: Uses flow matching instead of traditional diffusion models, achieving more efficient and stable audio generation by directly learning the transport path of probability distributions.
- Audio Infilling completion mechanism: Models singing voice synthesis as a conditional waveform completion task, using context segments to predict the target audio, naturally ensuring long-term coherence and timbre consistency.
- Explicit multimodal alignment: Enforces the temporal relationship among lyric text, MIDI notes, and acoustic features through a length regulator, eliminating rhythm deviations and pronunciation ambiguity caused by implicit alignment.
- Progressive two-stage training: Uses short-segment training to build score understanding ability and long-segment training to capture long-range breath control, ultimately balancing local precision and global naturalness.
SoulX-Singer's Project Address
- GitHub repository：https://github.com/Soul-AILab/SoulX-Singer
- HuggingFace model library：https://huggingface.co/Soul-AILab/SoulX-Singer
- arXiv technical paper：https://arxiv.org/pdf/2602.07803
SoulX-Singer's Application Scenarios
-
Virtual singer creation: The model can quickly create virtual idols with unique timbres, reducing the cost of signing and recording with real singers.
-
AI covers and derivative creation: Users can cover popular songs with any singer's timbre, enabling creative adaptations across languages and styles.
-
Music-assisted creation: Songwriters can quickly generate demos through MIDI input to verify the matching effect between melody and lyrics.
-
Audio content production: Batch-generate high-quality singing or chanting content for scenarios such as audiobooks, podcasts, and game voiceovers.
-
Personalized entertainment: Ordinary users upload their own voices to generate an exclusive AI singer to sing any song.