What is SongGeneration 2
SongGeneration 2 is a 4B-parameter music generation large model jointly open-sourced by Tencent and Tsinghua University. It adopts a hybrid LLM-diffusion architecture and hierarchical representation design to achieve commercial-grade music generation quality. The model achieves 8.55% PER in lyrics accuracy, surpassing commercial models such as Suno v5. SongGeneration 2 supports multiple languages including Chinese and English, complex multi-track arrangements, and can precisely control style through text descriptions and audio prompts. The model can run locally on consumer-grade hardware with 22GB VRAM and generate a complete song within 1 minute.
Main Features of SongGeneration 2
- High-Quality Song Generation: Generates complete songs up to 4 minutes and 30 seconds long with beautiful melodies and rich layered arrangements based on a hybrid LLM-diffusion architecture.
- Multilingual Lyrics Synthesis: Supports lyrics in multiple languages including Chinese, English, Spanish, and Japanese, achieving clear and accurate pronunciation with an extremely low phoneme error rate of 8.55%.
- Diverse Generation Modes: Flexibly outputs complete songs, instrumental music, pure vocals, or separated dual tracks of vocals and accompaniment.
- Precise Style Control: Precisely controls the generated style through text descriptions (gender, genre, mood, instruments) or 10-second audio prompts.
- Flexible Deployment Options: Supports local operation with 22GB VRAM and quick online experience via HuggingFace Space, and provides low-VRAM adaptation solutions.
Technical Principles of SongGeneration 2
- Hybrid LLM-Diffusion Architecture: The language model (LeLM) serves as the "composition brain" to coordinate the overall musical structure and performance details, while the diffusion model acts as a "high-fidelity renderer" to synthesize complex acoustic details under the guidance of the language model. The two collaborate with a division of labor to balance musicality and sound quality.
- Hierarchical Representation Modeling: Adopts a parallel modeling design. Mixed Tokens capture high-level melody and structural semantics, while Dual-Track Tokens separately model fine-grained acoustic variations of vocal and accompaniment tracks.
- Automated Aesthetic Evaluation: Builds a fine-grained evaluation framework based on 11,717 expert-annotated samples, injecting musicality prior knowledge into model training, and introduces a CFG strategy based on musicality labels during inference.
- Three-Stage Progressive Post-Training: Through SFT, narrows the data distribution to build a high-quality foundation; large-scale offline DPO uses 200,000 pairs of strict positive and negative samples to eliminate lyric hallucination; semi-online DPO periodically updates the model to push the upper limit of musicality to the extreme.
Project Address of SongGeneration 2
- GitHub Repository：https://github.com/tencent-ailab/songgeneration
- HuggingFace Model Library：https://huggingface.co/tencent/SongGeneration
Application Scenarios of SongGeneration 2
- Music Creation Assistance: Independent musicians can quickly generate demos to validate inspiration, reducing the threshold for arrangement and production costs.
- Video Soundtrack Production: Self-media creators can instantly generate customized background music according to video style and emotional needs.
- Game Audio Development: Game companies can batch-generate adapted theme music and sound effects for different scenes, characters, and storylines.
- Advertising and Marketing Content: Brands can generate exclusive jingles according to campaign tonality, enabling rapid iteration of audio assets.
- Education and Entertainment Applications: Music teaching platforms provide stylized accompaniment generation, and karaoke and social applications support users' personalized song creation.