What is ChildMandarin
ChildMandarin is a Mandarin speech dataset for children aged 3-5, jointly launched by the Beijing Academy of Artificial Intelligence (BAAI) and the Human Language Technology Lab (HLT Lab) of the College of Computer Science at Nankai University. The dataset contains 41.25 hours of speech data from 397 children, covering 22 provincial-level administrative regions in China, with a balanced gender distribution. The data was recorded with smartphones to ensure high-quality audio. The core feature of ChildMandarin is natural and authentic interaction, using a parent-guided conversation collection method to simulate natural communication scenarios. The dataset fills the gap in speech research on young children and promotes the development of child speech recognition, language development research, and intelligent speech interaction systems.
Main features of ChildMandarin
- Speech recognition: Provides a large amount of natural speech data from children aged 3-5 for automatic speech recognition (ASR) models, improving the accuracy and robustness of child speech recognition.
- Speaker verification: Supports speaker verification (SV) tasks, helping identify and distinguish different children's voices, and is used in scenarios such as child identity authentication.
- Language research: Provides data support for research on children's language development, helping develop children's language learning tools and interactive education systems.
Technical principles of ChildMandarin
- Data collection: Uses parent-guided conversations to simulate natural communication scenarios, ensuring the authenticity and naturalness of the speech data. Data collection covers 22 provincial-level administrative regions in China, ensuring the diversity of accents from different regions. Recording is done with smartphones (Android and iPhone) to ensure high audio quality, with a sampling rate of 16kHz and 16-bit precision.
- Data annotation: Manual annotation is performed by professional transcribers, including children's pronunciation, pauses, repetitions, and other natural language phenomena. Information such as the speaker's age, gender, birthplace, recording device, and accent level is annotated.
- Model training and evaluation: Uses a variety of ASR models (such as Transformer, Conformer, Paraformer) for training and evaluation, based on technologies such as CTC, AED, and RNN-T. Fine-tunes pre-trained models (such as HuBERT and Whisper) to use features learned from large-scale data to improve the performance of child speech recognition. Uses speaker embedding extraction models (such as x-vector, ECAPA-TDNN, and ResNet-TDNN) for speaker verification tasks to evaluate model performance on child speech.
- Dataset design: The dataset is divided into training, validation, and test sets to ensure the scientific validity and effectiveness of model training and evaluation. Based on diverse data collection and annotation, the dataset is ensured to cover child speech across different age groups, genders, regions, and accents.
Project address of ChildMandarin
- GitHub repository：https://github.com/flageval-baai/ChildMandarin
- HuggingFace model library：https://huggingface.co/datasets/BAAI/ChildMandarin
- arXiv technical paper：https://arxiv.org/pdf/2409.18584
Application scenarios of ChildMandarin
- Children's language learning tools: Develop intelligent voice-assisted tools to help children learn language pronunciation, vocabulary, and grammar, improving their language abilities.
- Interactive education systems: Provide voice interaction capabilities for children's education software and interactive learning platforms, making the learning process more vivid and interesting.
- Smart toy development: Enhance the speech recognition capabilities of smart toys to better understand children's voice commands and improve the interactive experience.
- Voice assistant optimization: Improve voice assistants' (such as smart speakers and phone assistants) recognition of and response to children's speech, making them more suitable for children to use.
- Children's health management: Monitor children's language development and health status to support early intervention.