DeepSeek-OCR - A vision-language model open-sourced by the DeepSeek team

What is DeepSeek-OCR

DeepSeek-OCR is DeepSeek a vision-language model launched by the team, focused on efficiently processing long text content through optical compression technology. The model consists of the DeepEncoder encoder and the DeepSeek3B-MoE decoder, and can significantly reduce activation memory and the number of visual tokens while maintaining high-resolution input. The model achieves up to 97% OCR accuracy at a 10x compression ratio, and can still maintain 60% accuracy at a 20x compression ratio. DeepSeek-OCR supports multiple resolution modes, is suitable for multilingual document processing, can parse complex content such as charts and chemical formulas, and provides an efficient solution for large-scale document processing.

Main features of DeepSeek-OCR

• Visual text compression: Efficiently compresses long text content through the visual modality, achieving a compression ratio of 7-20x.
• Multilingual OCR: Supports document recognition in nearly 100 languages, including Chinese, English, Arabic, Sinhala, and more.
• Deep parsing: Can parse complex content such as charts, chemical formulas, and geometric figures.
• Multi-format output: Supports layout-preserving Markdown format and layout-free free OCR format.

Technical principles of DeepSeek-OCR

•  Core architecture: DeepSeek-OCR consists of two parts:
  • DeepEncoder: Responsible for encoding the input image (document) into visual tokens.
  • DeepSeek-3B-MoE-A570M: As the decoder, responsible for decoding visual tokens into text.
• DeepEncoder: DeepEncoder is the core component of DeepSeek-OCR. Its design goal is to maintain low activation memory under high-resolution input while achieving a high compression ratio. It consists of the following parts:
  • Dual-tower structure：
    • SAM-base (80M): Based on window attention, mainly used to perceive local features, with low memory usage when processing high-resolution input.
    • CLIP-large (300M): Based on global attention, used to extract global semantic information. Since the input is compressed, the memory usage of the global attention part can be effectively controlled.
  • 16x convolutional compression layer: Between SAM and CLIP, DeepEncoder uses a 16x convolutional compression module. The module reduces the number of visual tokens from 4096 to 256 through two convolution layers (each with stride=2). This design significantly reduces the number of visual tokens and lowers memory usage without losing important information.
  • Multi-resolution support: DeepEncoder supports multiple resolution modes, including Tiny, Small, Base, Large, and Gundam. Each mode corresponds to a different input resolution and number of visual tokens. For example:
    • Tiny: 512x512 resolution, outputs 64 visual tokens.
    • Small: 640x640 resolution, outputs 100 visual tokens.
    • Base: 1024x1024 resolution, outputs 256 visual tokens.
    • Large: 1280x1280 resolution, outputs 400 visual tokens.
    • Gundam: Dynamic resolution, supports higher-resolution input, and further reduces activation memory through chunked processing.
• Decoder: DeepSeek-3B-MoE-A570M: The decoder is based on the DeepSeek-3B-MoE architecture and has 570M activated parameters. It is responsible for decoding the compressed visual tokens into text. The decoder converts visual tokens into text representations through a nonlinear mapping. The specific formula is: fdec​:Rn×dlatent​→RN×dtext​ where n is the number of visual tokens,N is the number of text tokens,dlatent​ and dtext​ are the dimensions of visual tokens and text tokens, respectively.

DeepSeek-OCR project address

• GitHub repository：https://github.com/deepseek-ai/DeepSeek-OCR
• HuggingFace model library：https://huggingface.co/deepseek-ai/DeepSeek-OCR
• Technical paper：https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSeek_OCR_paper.pdf

Application scenarios of DeepSeek-OCR

• Large-scale training data generation: automatically processes hundreds of thousands of document pages daily, providing massive high-quality training data for large language models and vision-language models.
• Enterprise-level document digitization: quickly and accurately converts various internal enterprise documents such as paper contracts and reports into searchable, editable digital formats.
• Academic research and literature processing: accurately parses complex content in academic papers, such as mathematical formulas, chemical formulas, and charts, converting it into structured machine-readable formats.
• Multilingual international document processing: easily handles the multilingual document processing needs encountered by multinational enterprises or organizations in globalized business operations.
• Financial and business intelligence analysis: can deeply parse charts in research reports and convert them into structured data, providing automated support for financial analysis and investment decisions.