LongCat-2.0-Preview - Meituan's Trillion-Parameter-Level Large Model Preview Version

What is LongCat-2.0-Preview

LongCat-2.0-Preview is a trillion-parameter-level large model preview version launched by Meituan. It adopts a MoE architecture, with total parameters reaching 1.6T and activated parameters at approximately 48B, supporting a 1M ultra-long context. The model completed the full training and inference process based on 50,000–60,000 domestic accelerator cards, with NVIDIA compute accounting for 0%. It is the industry's first to run a closed loop of trillion-level large model training and inference on domestic chips. The model is currently open for internal testing, providing 10 million free Tokens per day.

Main Features of LongCat-2.0-Preview

• Ultra-long context understanding: Supports 1M token long-text input, capable of processing massive content such as entire books, large codebases, and lengthy reports.
• Code generation and reasoning: Based on the MoE architecture, it precisely schedules expert networks and performs stably in programming, mathematical reasoning, and complex instruction-following tasks.
• Multi-turn dialogue and semantic retrieval: Through a sparse attention mechanism, it quickly locates key information in long dialogue histories and reduces redundant computation.
• Multimodal content understanding: Combined with Meituan's accumulation in local life scenarios, it supports in-depth parsing of complex business documents and mixed text-image content.

Technical Principles of LongCat-2.0-Preview

• MoE (Mixture of Experts) architecture: Of the total 1.6T parameters, only 48B are activated to participate in a single forward computation. Through gated routing, the most relevant expert modules are dynamically selected, reducing inference costs.
• N-gram Embedding: Some parameters originally located in the expert layers are moved forward to the Embedding layer, introducing phrase-level modeling capability so that high-frequency language patterns can be directly matched and hit, reducing layer-by-layer computation overhead.
• Sparse attention and cross-layer flow-aware indexing: Introduces a lightweight sparse attention mechanism, superimposed with cross-layer semantic path indexing, avoiding full O(n²) Attention computation and ensuring controllable inference latency under a 1M context.
• Self-developed deterministic operators: Targeting the domestic chip ecosystem, it self-develops FlashAttention backward gradients (FAG), Scatter-type operators, and GEMM modules, controlling performance loss to around 5% and ensuring the reproducibility of long-term training on a 10,000-card cluster.
• VRAM and communication optimization: Through the V-ZB algorithm, training peak VRAM is compressed to below 60GB, and the expert parallelism (EP), tensor parallelism (TP), and pipeline parallelism (PP) strategies are restructured to adapt to the bandwidth and VRAM constraints of domestic chips.

How to use LongCat-2.0-Preview

• Apply for internal testing eligibility: VisitLongCat Open Platform, click the internal testing application entry on the official website, fill in basic information, and submit for review.
• Wait for review approval: The team will conduct eligibility review after the application is submitted, and after approval, notify the user by email or SMS to activate permissions.
• Log in to the platform to obtain quota: Log in to the LongCat platform with the approved account, and the system will automatically issue a free calling quota of 10 million Tokens per day.
• Choose an access method: Users can experience it directly through the Web chat interface, or obtain an API Key according to the documentation guidelines to integrate the model into their own applications and workflows.
• Start long-text tasks: Enter long documents, codebases, or complex instructions in the dialog box or API request, and the model will complete inference based on the 1M context window and MoE expert network and return results.