SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

FireRedASR2S - Xiaohongshu's Open-Source Speech Recognition Model

Author: AI小集 · Original source: AI工具集

What is FireRedASR2S

FireRedASR2S is an industrial-grade end-to-endspeech recognitionmodel open-sourced by Xiaohongshu's Super Intelligence-AudioLab, integrating four SOTA modules: ASR, VAD, language identification, and punctuation prediction. The model supports Mandarin Chinese and 20+ dialects, English, code-switching, and lyrics recognition, with a character error rate as low as 2.89% for Mandarin Chinese and an average of 11.55% for dialects, comprehensively outperforming competitors such as Doubao-ASR,Qwen3-ASRThe system supports one-click local deployment without external APIs and has been deployed at scale in high-frequency scenarios such as Xiaohongshu voice comments and voice search.

FireRedASR2S

Main Features of FireRedASR2S

  • Speech Recognition (FireRedASR2): Supports Mandarin Chinese, 20+ dialects/accents, English, Chinese-English code-switching, and lyrics recognition, offering two architecture versions: LLM and AED. The AED version supports character-level timestamps and confidence output.
  • Voice Activity Detection (FireRedVAD): The model can detect speech/singing/music, supports 100+ languages, provides both streaming and non-streaming modes, and achieves an F1 score of 97.57%.
  • Language Identification (FireRedLID): Supports recognition of 100+ languages and 20+ Chinese dialects, with an accuracy of 97.18%, significantly outperforming open-source solutions such as Whisper.
  • Punctuation Prediction (FireRedPunc): The model automatically adds Chinese and English punctuation, with an average F1 score of 78.90%, greatly improving the readability of transcribed text.

Technical Principles of FireRedASR2S

  • Speech Recognition (FireRedASR2): The model adopts two architectures: Encoder-Adapter-LLM and Attention-based Encoder-Decoder. The LLM version uses large language model capabilities to achieve end-to-end speech understanding, while the AED version optimizes computational efficiency on the encoder-decoder framework, fusing speech and text representations through an adapter layer, and supports timestamp and confidence output.
  • Voice Activity Detection (FireRedVAD): Based on the DFSMN deep feedforward sequential memory network, it models audio temporal features. It determines speech onset and offset through smoothing windows and thresholds, distinguishes audio events such as speech, singing, and music, and supports streaming processing to meet real-time requirements.
  • Language Identification (FireRedLID): Reuses the FireRedASR2 encoder to extract speech representations and trains a classifier to predict language labels. Leveraging large-scale multilingual data pretraining, it establishes a cross-lingual shared representation space, enabling high-accuracy recognition of 100+ languages and dialects.
  • Punctuation Prediction (FireRedPunc): Based on the BERT architecture, it takes unpunctuated text as input and predicts the punctuation type at each position. Through fine-tuning on multi-domain Chinese and English data, it learns text semantics and syntactic structure and automatically inserts appropriate punctuation marks.

FireRedASR2S Project Address

  • GitHub Repository：https://github.com/FireRedTeam/FireRedASR2S
  • HuggingFace Model Hub：https://huggingface.co/collections/FireRedTeam/fireredasr2s

Application Scenarios of FireRedASR2S

  • Content Community Interaction: Supports functions such as Xiaohongshu voice comments and voice search, allowing users to participate in community interaction with diverse voices such as dialects and singing, enhancing the platform's sense of authenticity and fun.
  • Social and Communication: Empowers scenarios such as voice private messages and voice New Year greetings, enabling natural and fluent voice input and real-time transcription, lowering communication barriers and improving emotional transmission efficiency.
  • Content Creation and Production: Supports creative tools such as voice note publishing, live stream subtitle generation, and automatic video subtitles, helping creators efficiently produce multimedia content.
  • Enterprise-Level Services: Suitable for B2B scenarios such as meeting transcription, intelligent customer service, and telephone analysis; its private deployment capability meets the data security and compliance requirements of industries such as finance and healthcare.