What is LongCat-Flash-Lite
LongCat-Flash-Lite is a new generation of highly efficient large language model launched by Meituan. The model adopts an innovative Mixture of Experts (MoE) + N-gram embedding architecture, with a total parameter count of 68.5 billion, activating only about 2.9 to 4.5 billion parameters per inference, balancing powerful capabilities with extremely high efficiency. The model supports a 256K ultra-long context and performs excellently in core evaluations such as agent tasks, code generation, and mathematical reasoning. In tool calling and programming, its performance reaches the top level among models of the same activation scale. The LongCat API can provide a generation speed of 500-700 token/s. Through dedicated system optimization, the model's inference speed is significantly improved.
Main Features of LongCat-Flash-Lite
-
Text Generation: The model supports multi-turn dialogue interaction and can generate fluent and coherent natural language replies.
-
Tool Calling: The model has function calling capabilities and can autonomously use external tools to complete complex tasks.
-
Code Generation: It is skilled at programming tasks and can write, understand, and debug code in multiple programming languages.
-
Long Context Processing: The model supports a 256K ultra-long context and can handle tasks such as long document analysis.
Technical Principles of LongCat-Flash-Lite
- MoE + N-gram Embedding Architecture: LongCat-Flash-Lite adopts a Mixture of Experts (MoE) architecture, with a total parameter count of 68.5B and only 2.9B~4.5B parameters activated. Unlike traditional MoE that uses FFN as experts, the model innovatively introduces N-gram embedding tables (NE) to replace some experts, forming a hybrid "MoE + NE" architecture.
- N-gram Embedding Table Mechanism: The N-gram embedding table converts some computationally intensive operations into table lookup operations by precomputing and storing embedding vectors of N-gram combinations. The core advantage of the mechanism is that the memory access pattern of embedding lookup is more regular and has lower latency, greatly reducing data movement overhead compared to FFN forward propagation. Through systematic scaling experiments, the research team determined key factors such as the optimal integration timing of the embedding table, parameter budget allocation, hash collision mitigation strategies, hyperparameter configurations, and embedding initialization methods.
- Inference Efficiency Optimization System: To fully leverage the performance advantages of the N-gram embedding table, the team launched a dedicated inference optimization system. The system contains two core components: N-gram Cache, which reduces the overhead of repeated embedding lookups through intelligent caching strategies; and Synchronized Kernels, customized CUDA kernels that achieve pipeline parallelism between embedding lookups and other computations.
- Long Context Extension Technology: The model adopts the YaRN (Yet another RoPE extension method) method to extend the context length to 256K. YaRN improves Rotary Position Embedding (RoPE) by dynamically adjusting the rotation angle and temperature scaling factor of positional encoding, enabling the model to effectively process longer input sequences without additional training based on the shorter context used during training (usually 4K-8K), while maintaining sensitivity and stability to relative positional relationships.
LongCat-Flash-Lite Project Address
- HuggingFace Model Repository：https://huggingface.co/meituan-longcat/LongCat-Flash-Lite
- arXiv Technical Paper：https://arxiv.org/pdf/2601.21204
Application Scenarios of LongCat-Flash-Lite
- Intelligent Customer Service and Dialogue Systems: The model supports multi-turn dialogue and tool calling, making it suitable for intelligent customer service scenarios in industries such as aviation, retail, and telecommunications, where it can autonomously query information, process orders, and resolve user issues.
- Code Development and Assisted Programming: It performs excellently on code benchmarks such as SWE-Bench and can be used for code generation, bug fixing, code review, and automated programming tasks, improving development efficiency.
- Long document processing and analysis: The model is suitable for scenarios requiring the processing of large amounts of text, such as legal contract analysis, academic paper study, batch processing of financial reports, and understanding long video scripts.
- Agents and automated workflows: The model's powerful Agentic capabilities can serve as a core engine to drive automated workflows and execute multi-step tasks, such as data collection, report generation, and system operations and maintenance.
- Multilingual content processing: The model performs in a balanced way in multilingual evaluations such as Chinese and English, and is suitable for globalization business scenarios such as cross-language content generation, localization translation, and multilingual knowledge question answering.