SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Youtu-LLM - A lightweight language model open-sourced by Tencent Youtu Team

Author: AI小集 · Original source: AI工具集

What is Youtu-LLM

Youtu-LLM is a lightweight language model open-sourced by the Tencent Youtu team, with a parameter scale of 1.96 billion. Designed for agent tasks, it possesses strong "native agent capabilities" and surpasses models of the same scale or even larger in multiple tasks. The model adopts a compact architecture and a 128K long context window, supports long-horizon task processing, and optimizes the vocabulary for the STEM field to improve reasoning efficiency. The model has been fully open-sourced, supports Base and Instruct versions, and provides fine-tuning tools for developers to use.

Youtu-LLM

Main Features of Youtu-LLM

  • Powerful agent capabilities: Possesses native agent capabilities, can autonomously plan tasks, execute operations, and adjust strategies based on feedback, suitable for complex task processing.
  • Efficient reasoning and long context support: Adopts a compact architecture, supports a 128K context window, suitable for long-horizon tasks such as complex code repair and multi-hop research.
  • Optimized STEM vocabulary: Designed for professional fields such as mathematics and code, improving token compression rate and reasoning efficiency.
  • Systematic pretraining curriculum: Pretrained from scratch, gradually improving model capabilities through three stages of learning: common sense, STEM, and agent.
  • High-quality agent trajectory data: Introduces a large amount of agent trajectory data, covering scenarios such as mathematical reasoning, code repair, and deep research, improving the model's performance on agent tasks.
  • Open source and flexible deployment: Fully open-sourced, supports Base and Instruct versions, provides fine-tuning tools, making it convenient for developers to deploy in resource-constrained scenarios.

Technical Principles of Youtu-LLM

  • Compact architecture and long context: Adopts a Dense MLA architecture, supports a 128K context window, suitable for long-horizon task processing while maintaining efficient reasoning capabilities.
  • Optimized tokenizer design: A 128K vocabulary specially designed for the STEM field, improving reasoning efficiency in professional fields such as mathematics and code.
  • Three-stage pretraining curriculum: Pretrained from scratch, divided into the common sense stage, STEM-focused stage, and agent mid-training stage, gradually improving the model's reasoning and planning capabilities.
  • Agent trajectory synthesis: Introduces high-quality agent trajectory data, covering scenarios such as mathematical reasoning, code repair, and deep research, strengthening the model's performance on agent tasks.
  • Innovative training paradigm: Through curriculum-style training of "common sense → STEM → agent", the model internalizes capabilities such as planning, execution, and reflection, achieving lightweight yet native intelligence.

Youtu-LLM Project Address

  • Project official website：https://youtu-tip.com/#llm
  • Github repository：https://github.com/TencentCloudADP/youtu-tip/tree/master/youtu-llm

Application Scenarios of Youtu-LLM

  • Code assistant: Can understand complex codebase structures, locate and fix errors in code, write code that conforms to project specifications, and automate software engineering tasks.
  • Research assistant: Suitable for literature review and information integration, supports multi-hop question answering and knowledge reasoning, can automatically generate research reports, and assist academic writing.
  • General-purpose agent: Can automate daily tasks, enable multi-tool collaborative work, decompose and execute complex tasks, and be used as a personal AI assistant.
  • Edge deployment: Due to the smaller number of parameters, it is suitable for running on consumer-grade GPUs, mobile devices, and edge devices, providing low-latency, low-cost inference services, and is suitable for local deployment in privacy-sensitive scenarios.
  • Multi-hop reasoning and summarization: Can solve complex problems requiring multi-step reasoning, such as deep association analysis, causal reasoning, etc.
  • Knowledge-intensive tasks: Efficiently handles problems that depend on large amounts of structured knowledge, such as enterprise knowledge base question answering and in-depth parsing of technical documents.