EXAONE Deep is a series of reasoning-enhanced language models built and released by LG AI Research, with the official repository at LG-AI-EXAONE/EXAONE-Deep. According to the official description, this series covers three parameter scales, 2.4B, 7.8B, and 32B, and demonstrates strong capabilities on various reasoning tasks such as mathematics and programming. The officially published evaluation results show: EXAONE Deep 2.4B outperforms other models of the same scale; EXAONE Deep 7.8B not only outperforms open-weight models of the same scale but also outperforms the proprietary reasoning model OpenAI o1-mini; EXAONE Deep 32B is competitive with leading open-weight models.
Main capabilities and uses: This series is positioned as reasoning-enhanced language models, and the official documentation covers sections such as performance, quick start, quantized models, local running, deployment, and usage guide. The models are trained with optimization for reasoning tasks, and the official recommendation is to follow the usage guide to achieve the expected performance.
How to use: Using EXAONE Deep models requires installing transformers>=4.43.1, and the official recommendation is to use the latest version. The official documentation provides example code and explains that in all examples, other-scale models can be used by replacing 7.8B with 32B or 2.4B. The models have been uploaded to the Hugging Face and Ollama model repositories.
Quantization and local running: The official release provides AWQ-quantized weights, quantized using the AutoAWQ library; loading the AWQ-quantized version requires installing autoawq>=0.2.8. It also provides BF16 format weights as well as quantized weights such as Q8_0, Q6_K, Q5_K_M, Q4_K_M, and IQ4_XS. Users can run the model in chat mode via llama.cpp, and the official recommendation is to explicitly set the chat-template to correctly handle the reasoning steps; when using EXAONE Deep 32B at BF16 precision, it may be necessary to download all shard files and merge them before running. It can also be run via Ollama, for example exaone-deep:7.8b uses Q4_K_M quantization; users can also write their own Modelfile and convert it to an Ollama model. In addition, it can be run locally in LM-Studio, where "Reasoning Section Parsing" needs to be set to <thought> and </thought>, and set the"Template (Jinja)" is set to the template for EXAONE 3.5.
Deployment: The model has been integrated into multiple deployment frameworks. TensorRT-LLM has supported EXAONE language models since EXAONE 3.0, and it is officially recommended to use TensorRT-LLM for optimal performance. When converting to the TensorRT-LLM format, you may need to set the environment variable TRTLLM_DISABLE_UNIFIED_CONVERTER=1; TensorRT-LLM also supports AWQ. To use vLLM, you need to install vllm>=0.6.0, and you can send curl requests; if you want to serve a GGUF-quantized model with vLLM, you need to refer to the vLLM GGUF documentation. When using SGLang, if it is the EXAONE Deep 2.4B model, you need to install sglang>=0.3.6 and use the --attention-backend triton option; the official team also mentioned that they are fixing the incompatibility between flashinfer and EXAONE Deep 2.4B through a PR.
Usage Guide: The official recommendation is to ensure that the model starts its reasoning steps with <thought>\n, which can be achieved by using tokenizer.apply_chat_template() and setting add_generation_prompt=True; the model output quality may degrade when this starting marker is omitted. Since reasoning steps usually contain a large number of tokens, in multi-turn scenarios it may be necessary to remove previous reasoning steps, and the provided tokenizer will handle this automatically. The official recommendation is to avoid using a system prompt and instead build instructions into the user prompt. Additional instructions help the model reason more deeply; for example, for math problems you can use "Please reason step by step, and put your final answer within \boxed{}.". In the official evaluation, temperature=0.6 and top_p=0.95 were used for generation, and it is recommended to test multiple times to accurately evaluate expected performance. It is also strongly recommended to use a repetition penalty no greater than 1.0 to obtain better generation quality.
Limitations: The official team points out that this language model has certain limitations and may occasionally generate inappropriate responses. The model generates content based on token output probabilities, determined by what is learned from the training data. Although every effort has been made to remove from the training datapersonal, harmful, and biased information, it may still contain problematic content and lead to undesirable responses; the generated text does not represent the views of LG AI Research. It may generate responses containing personal, harmful, or otherwise inappropriate information; it may generate biased responses related to age, gender, race, etc.; the generated content is highly dependent on training data statistics and may produce sentences with semantic or grammatical errors; the model does not reflect the latest information, and responses may be incorrect or contradictory. LG AI Research is committed to reducing potential risks, and users must not engage in any malicious activities that may induce inappropriate outputs violating LG AI's ethical principles when using the EXAONE language model (for example, inputting illegal information).
License and Support: The model is licensed under the EXAONE AI Model License Agreement 1.1 - NC. The LG AI Research technical support email is contact_us@lgresearch.ai.
