Chroma 1.0 - FlashLabs' Open-Source Real-Time End-to-End Speech Dialogue Model

What is Chroma 1.0

Chroma 1.0 is FlashLabs' first open-source real-time end-to-end speech dialogue model, combining low-latency interaction, high-fidelity personalizedvoice cloningand strong dialogue capabilities. By tightly coupling speech understanding and generation, the model adopts a 1:2 text-audio token scheduling strategy to achieve sub-second latency output. With just a few seconds of reference audio, it can highly reproduce the speaker's timbre characteristics, with speaker similarity 10.96% higher than the human baseline. The model has only 4B parameters and performs excellently in reasoning and spoken dialogue tasks, balancing efficiency and performance.

Main Features of Chroma 1.0

• Real-time speech interaction: Supports low-latency speech dialogue with end-to-end latency under 1 second, suitable for real-time interaction scenarios.
• High-fidelity voice cloning: The model requires only a few seconds of reference audio to achieve highly similar personalized speech synthesis, with timbre reproduction 10.96% higher than the human baseline.
• Powerful dialogue capabilities: The model possesses understanding, reasoning, and spoken dialogue abilities, supporting complex dialogue tasks such as story logic and fact judgment.
• Streaming generation: Adopts a streaming output architecture, supports continuous dialogue, and generates faster than real-time playback (RTF of 0.43).
• Multimodal fusion: Combines text and audio inputs, preserving paralinguistic information such as speech rhythm and intonation to achieve more natural interaction.

Technical Principles of Chroma 1.0

• Tightly coupled speech understanding and generation: Tightly integrates the speech understanding module (Chroma Reasoner) with the speech generation modules (Chroma Backbone, Chroma Decoder, and Chroma Codec Decoder), enabling low-latency streaming output through semantic state representation.
• 1:2 text-audio token scheduling strategy: During generation, each text token corresponds to 2 audio codebook tokens, enabling synchronous audio and text generation and significantly reducing latency.
• High-fidelity voice cloning: By embedding reference audio and corresponding text into the input sequence, the model can learn and reproduce the timbre characteristics of a specific speaker.
• Multimodal attention mechanism: Uses cross-modal attention and time-aligned multimodal rotary position embedding (TM-RoPE) to ensure temporal alignment between speech and text, enhancing the naturalness of dialogue.
• Discrete acoustic representation and causal CNN: The model uses discrete acoustic codebook representation for speech, performs waveform reconstruction through a causal convolutional neural network (Causal CNN), and supports real-time streaming output.

Project Address of Chroma 1.0

• GitHub Repository：https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma
• HuggingFace Model Library：https://huggingface.co/FlashLabs/Chroma-4B
• arXiv Technical Paper：https://arxiv.org/pdf/2601.11141

Application Scenarios of Chroma 1.0

• Intelligent customer service: Provides users with real-time speech interaction services, quickly responds to customer needs, offers personalized voice answers, and improves user experience.
• Voice assistant: Integrated into smart homes and smart devices, it controls devices through natural voice dialogue for convenient operation.
• Virtual anchor: Used in news broadcasting, live streaming, and other fields to generate high-fidelity speech, simulate specific anchor styles, and increase content diversity.
• Voice content creation: Assists content creators in generating high-quality voice content, such as audiobooks, voice stories, etc., improving creation efficiency.
• Education: Provides language learners with personalized voice conversation practice, real-time feedback on pronunciation and intonation, and improves language learning outcomes.