What is Yi-Lightning
Yi-Lightning is the latest flagship model released by 01.AI. It achieved notable results on the internationally authoritative blind test leaderboard LMSYS, surpassing well-known Silicon Valley models OpenAI's GPT-4o-2024-05-13 and Anthropic Claude 3.5 Sonnet, ranking sixth in the world and first in China. This achievement marks the first time a Chinese large model has surpassed OpenAI's GPT-4o on the global competitive stage, demonstrating China's strength and progress in the field of artificial intelligence.
The Yi-Lightning model also demonstrated excellent performance on multiple sub-leaderboards. On the Chinese sub-leaderboard, it surpassed other high-quality models domestic and abroad, tying with models such as o1-mini for second place in the world. On the multi-turn conversation sub-leaderboard, Yi-Lightning ranked third, and on the mathematics and code sub-leaderboards, it achieved third and fourth place, respectively.
Yi-Lightning also achieved breakthroughs in inference speed and cost. Compared with the previous-generation flagship model Yi-Large, Yi-Lightning's maximum generation speed increased by nearly 40%, and the first-packet time was halved. Yi-Lightning's inference cost was further reduced, requiring only 0.99 yuan per million tokens, approaching the industry's lowest price.
Main features of Yi-Lightning
- Inference speed and cost: Yi-Lightning has significantly improved inference speed compared with the previous-generation flagship model Yi-Large, with maximum generation speed increased by nearly 40%. Inference cost was also further reduced, requiring only 0.99 yuan per million tokens.
- AI 2.0 digital human solution: 01.AI launched an AI 2.0 digital human solution based on the Yi-Lightning model, focusing on scenarios such as retail and e-commerce. This solution includes character large models, livestreaming voice large models, e-commerce script large models, and more, with capabilities such as motion training, expression generation, multilingual and emotional expression, and intelligent dialogue. In practical application, after a certain hotel and travel enterprise integrated it, GMV increased by 170% compared with before.
- All-industry solutions: The Yi-Lightning model is also applied in 01.AI's all-industry solutions. In addition to the base model, these solutions are also paired with practical tools such as RAG and Function Calling. They have already been implemented in fields such as retail, healthcare, education, and logistics, involving application scenarios such as AI search, AI productivity tools, and AI intelligent inspection.
- Model architecture innovation: Yi-Lightning adopts a Mixture of Experts (MoE) architecture. During model training, it introduced a hybrid attention mechanism and a dynamic Top-P routing mechanism. This innovative attempt provides a standardized base model with a higher starting point, enabling faster, more efficient, and lower-cost completion of custom model training.
- Ultra-fast inference: Yi-Lightning's inference speed is very fast. Based on the dynamic Top-P routing mechanism, it can dynamically and automatically select the most suitable combination of expert networks according to task difficulty, balancing inference cost and model performance.
- Multi-stage training: Yi-Lightning's training process adopts a multi-stage training model. In the early stage, it focuses on data diversity, and in the later stage, it focuses on richer and more knowledge-intensive data. This training method helps the model absorb knowledge from different stages and ensures training speed and stability by adjusting batch size and learning rate (LR).
Technical principles of Yi-Lightning
- MoE Mixture of Experts model architecture: Yi-Lightning adopts the Mixture of Experts (MoE) architecture, which combines multiple expert networks (Experts) to handle different tasks, enabling the model to dynamically select which expert networks to activate based on task difficulty, balancing inference cost and model performance. During training, all expert networks are activated; during inference, the model selectively activates the expert networks that better match the task.
- Hybrid Attention: Yi-Lightning optimizes the hybrid attention mechanism by replacing traditional Full Attention with Sliding Window Attention only in some layers of the model, reducing computational cost while maintaining efficient processing capability for long-sequence data.
- Cross-Layer Attention (CLA): Yi-Lightning introduces the cross-layer attention mechanism, which supports the model in sharing Key and Value heads across different layers, reducing storage resource requirements and improving the model's inference efficiency.
- Dynamic Top-P Routing: Yi-Lightning dynamically and automatically selects the most suitable combination of expert networks based on task difficulty, without manual intervention, enabling the model to more intelligently adapt to various task requirements and achieve extremely fast inference.
Yi-Lightning project address
- Project official website：platform.lingyiwanwu.com
Yi-Lightning application scenarios
- Translation scenarios: Yi-Lightning performs excellently in the translation field, capable of handling language understanding and generation, cross-lingual capabilities, context awareness, and more, providing high-quality translation services. In comparisons with multiple models, Yi-Lightning's translation ability was intuitively demonstrated, with precise word choice and literary flair.
- Retail and e-commerce livestreaming scenarios: The AI 2.0 digital human solution launched by 01.AI focuses on scenarios such as retail and e-commerce; based on the technical support provided by Yi-Lightning, it implements functions such as bullet comment interaction, product information extraction, and real-time script generation. After integrating Yi-Lightning, the digital human's real-time interaction effect is better, the scripts are smoother, and the replies are more accurate.
- Enterprise-level solutions: Yi-Lightning is also applied in the enterprise-level solutions under 01.AI's To B strategy, providing customized AI services for enterprises and helping them improve efficiency and revenue.
- Multilingual processing: On the Chinese sub-leaderboard, Yi-Lightning demonstrated strong Chinese processing ability and is not inferior compared with top international models.
- Mathematics and code generation: On the mathematics and code sub-leaderboards, Yi-Lightning achieved third and fourth place respectively, demonstrating its strong capabilities in these fields.
- Long questions and difficult questions: Yi-Lightning also performs outstandingly in handling long questions and difficult questions, achieving an excellent fourth place in the world in both, proving its ability to solve complex problems.