What is HY-1.8B-2Bit
HY-1.8B-2Bit is the first industry-grade 2Bit on-device large model launched by Tencent Hunyuan. The model is released based on the 1.8B model compressed through quantization-aware training (QAT). HY-1.8B-2Bit has an equivalent parameter count of only 0.3B, a size of only 300MB, and a memory footprint of 600MB, smaller than common mobile applications. While maintaining the full thinking capability of the original model, generation speed is increased by 2-3 times. It has been adapted to mobile platforms such as Arm SME2 and can run efficiently on edge devices such as phones, earphones, and smart homes, achieving a breakthrough in 'small but strong' on-device AI deployment.
Main features of HY-1.8B-2Bit
-
On-device intelligent inference: The model supports offline deployment and operation of large language models on consumer-grade hardware such as phones, earphones, and smart homes, without relying on cloud services.
-
Full thinking capability: It inherits the original model's flexible switching mechanism between long and short chains of thought, automatically selecting concise or detailed reasoning paths according to task complexity.
-
Efficient text generation: On real on-device devices, it achieves a 2-3 times generation speed improvement compared with the original precision model, greatly reducing user waiting time.
-
Privacy and security protection: The model runs entirely locally, ensuring that user data does not leave the device, meeting the security needs of offline deployment and highly private scenarios.
Technical principles of HY-1.8B-2Bit
- Quantization-aware training (QAT): Unlike traditional post-training quantization (PTQ), which directly compresses the model after training is complete and causes huge precision loss, QAT simulates the forward propagation effect of 2Bit quantization during the training process, enabling model weights to actively adapt to low-precision constraints and significantly restoring model performance after quantization.
- Extreme 2Bit compression and equivalent parameter reduction: By compressing 32-bit floating-point weights into 2Bit representation (only 4 discrete states), the model's actual size is reduced from the GB level of the original 1.8B parameters to 300MB, with an equivalent parameter count of only 0.3B, achieving a 6x storage compression ratio while maintaining mathematical, coding, and scientific reasoning capabilities comparable to 4Bit PTQ.
- Data and training strategy optimization: Three methods—data optimization, elastic stretch quantization, and training strategy innovation—are used together to improve the model's all-subject capabilities. Among them, elastic stretch quantization dynamically adjusts the quantization range of the weight distribution to minimize precision loss caused by extreme values.
- On-device hardware adaptation: It provides GGUF-INT2 format weights and BF16 pseudo-quantized weights, and is deeply optimized for the Arm SME2 instruction set, enabling the model to achieve efficient and stable inference on mobile devices that support this technology.
Project addresses of HY-1.8B-2Bit
- GitHub repository：https://github.com/Tencent/AngelSlim
- HuggingFace model library：
- https://huggingface.co/AngelSlim/HY-1.8B-2Bit-GGUF
- https://huggingface.co/AngelSlim/HY-1.8B-2Bit
- Technical paper：https://huggingface.co/AngelSlim/HY-1.8B-2Bit/blob/main/AngelSlim_Technical_Report.pdf
Application scenarios of HY-1.8B-2Bit
- Smartphone assistant: As a built-in AI assistant, it enables offline voice interaction, text generation, and intelligent Q&A, and can handle daily tasks such as schedule management and email drafting without an internet connection, protecting user privacy while reducing data consumption.
- Smart wearable devices: The model can be deployed on low-power devices such as smart earphones and watches, supporting offline speech-to-text, real-time translation, and health consultation, solving the pain points of limited computing power and battery life sensitivity in wearable devices.
- Smart home central control: It runs locally on IoT devices such as smart speakers and home appliance controllers, enabling offline voice command recognition, scene linkage control, and personalized recommendations, improving response speed and ensuring home data security.
- In-vehicle edge computing: Applied in in-vehicle systems and autonomous driving assistance devices to ensure service continuity in unstable network scenarios.
- Enterprise private deployment: The model can meet the needs of industries with extremely high data compliance requirements, such as finance, healthcare, and government affairs, enabling sensitive document processing and professional knowledge Q&A on local servers or dedicated terminals, eliminating the risk of data leakage.