What is GTSinger
GTSinger is a large-scale open-source high-quality singing voice dataset launched by a research team at Zhejiang University, designed to support diverse singing voice tasks. GTSinger contains 80.59 hours of singing voice recorded in professional studios, covering nine different languages (Chinese, English, Japanese, Korean, Russian, Spanish, French, German, and Italian), performed by 20 professional singers, providing rich diversity in timbre and style. GTSinger focuses on the control and modeling of singing techniques, providing control groups and phoneme-level annotations for six commonly used singing techniques. GTSinger provides real musical scores, which is helpful for actual music creation. The dataset includes manual phoneme alignment, global style labels, and paired reading data, adapting to various singing voice tasks.
Main Features of GTSinger
- Multilingual Singing Voice Dataset: GTSinger contains singing voices in nine different languages, providing diverse timbres and styles, and supporting cross-lingual singing voice synthesis and analysis.
- Singing Technique Control: The dataset provides control groups and phoneme-level annotations for six commonly used singing techniques, allowing researchers to better model and control techniques in singing voices.
- Real Musical Score Support: Provides real musical scores matching the singing voices, which is very helpful for applying singing voice synthesis technology to actual music creation.
- Multi-Task Adaptation: GTSinger is designed to support multiple singing voice tasks, including singing voice synthesis, technique recognition, style transfer, and speech-to-singing conversion.
- Benchmark Testing: Provides benchmark tests to evaluate the performance and applicability of the dataset on different singing voice tasks.
Technical Principles of GTSinger
- High-Quality Audio Recording: The GTSinger dataset is constructed by recording professional singers in professional studios, ensuring the high quality of the audio data.
- Phoneme Alignment and Annotation: Based on music information retrieval technologies such as MFA and Praat, phoneme alignment and annotation are performed to achieve precise phoneme-level control.
- Singing Technique Annotation: Based on expert listening perception and audio analysis technology, singing techniques in the singing voice are annotated, facilitating model learning and control.
- Musical Score Generation: Combining audio signal processing technology and music theory knowledge, pitch information is extracted from the singing voice and converted into MIDI-form musical scores, which are then adjusted by experts into real musical scores.
- Dataset Construction and Validation: Based on manual review and subsequent processing, the quality and applicability of the dataset are ensured, including semantic segmentation of audio clips and handling of silent regions.
GTSinger Project Address
- Project Official Website：gtsinger.github.io
- GitHub Repository：https://github.com/GTSinger/GTSinger
- HuggingFace Model Library：https://huggingface.co/datasets/GTSinger/GTSinger
- arXiv Technical Paper：https://arxiv.org/pdf/2409.13832
Application Scenarios of GTSinger
- Singing Voice Synthesis: Based on the singing voice samples and technique annotations in the dataset, develop systems that synthesize high-quality singing voices with specific techniques and styles.
- Singing Technique Recognition: Analyze phoneme-level technique annotations in singing voices, and train models to recognize and classify different singing techniques.
- Singing Voice Style Transfer: Convert singing voices of one style to another style, for example, converting the singing voice of pop songs to a classical style.
- Speech-to-Singing Conversion(Speech-to-Singing, STS): Convert ordinary speech into melodic singing, used in speech synthesis and music creation.
- Music Education: Based on real musical scores and singing samples in the dataset, develop music teaching tools to help students learn and practice singing techniques.