What is Reka Flash 3
Reka Flash 3 is an open-source reasoning model released by Reka AI, with 2.1 billion parameters. It supports multimodal input, including text, images, video, and audio, and can handle a context length of up to 32k tokens, making it suitable for general conversation,coding assistance, instruction following, and other scenarios. The full-precision model size is 39GB (fp16), and it can be compressed to 11GB after 4-bit quantization, making it convenient for on-device deployment.
Main features of Reka Flash 3
- Multimodal input support: Reka Flash 3 can process multiple types of input data, including text, images, video, and audio.
- Efficient reasoning capability: Reka Flash 3 supports up to 32k tokens of context length, and can handle longer documents and complex tasks, such as long papers, code snippets, or complex multi-step problems.
- Multilingual support: Reka Flash 3 performs excellently in multilingual aspects, mainly focusing on English, with a COMET score of 83.2 on WMT’23, meeting the needs of users in different languages.
Technical principles of Reka Flash 3
- Modular encoder-decoder architecture: Reka Flash 3 adopts a modular encoder-decoder architecture. The encoder is responsible for processing multiple types of input data, including text, images, video, and audio, converting them into high-dimensional vector representations. The decoder generates text output based on the encoder's output.
- Reinforcement learning training: The training of Reka Flash 3 combines publicly available datasets and synthetic datasets, using the REINFORCE Leave One-Out (RLOO) method for reinforcement learning. This enables the model to achieve a balance between capability and efficiency, and to better understand and generate high-quality content.
- Budget forcing mechanism: Through specific
<reasoning>tags, users can limit the model's reasoning steps, ensuring that the model maintains efficient and consistent performance under limited computational resources.
Project address of Reka Flash 3
- HuggingFace Model Library：https://huggingface.co/RekaAI/reka-flash-3
Application scenarios of Reka Flash 3
- General conversation: Reka Flash 3 can conduct natural and fluent conversations, providing services such as companionship and information inquiries.
- Coding assistance: The model can help developers write code, debug code issues, and provide code optimization suggestions.
- Instruction following: Reka Flash 3 can understand and execute complex instructions and complete various tasks, such as generating reports, analyzing data, and more.
- Web search assistance: Reka Flash 3 can call web search functionality to obtain additional information to assist with answers.