Unlimited-OCR - Baidu's Open-Source End-to-End Long Document OCR Model

What is Unlimited-OCR

Unlimited-OCR is an end-to-end long document OCR model launched by Baidu. Through the Reference Sliding Window Attention mechanism, it compresses the decoder KV cache from linear growth to a constant, enabling transcription of dozens of pages of documents in a single forward pass. The model is based on a 3B total parameter MoE architecture and achieves end-to-end SOTA on OmniDocBench v1.6 with an overall score of 93.92%, reaching an inference speed of 5580 TPS. The code and weights have been fully open-sourced.

Main Features of Unlimited-OCR

• Ultra-long document recognition in one pass: Supports single forward-pass transcription of PDF documents from 2 pages to 40+ pages, without the need for page-by-page for-loop processing.
• Multi-type document parsing: Covers 9 layout types including PPT, academic papers, books, color textbooks, exam papers, magazines, newspapers, notes, and research reports.
• High-precision content extraction: End-to-end output of text, formulas, tables, and reading order across the full pipeline, with formula CDM reaching 95.79% and table TEDS reaching 93.32%.
• Dual-resolution visual encoding: Base mode 1024×1024 is used for multi-page long documents, while Gundam mode with dynamic resolution is used for single-page high-precision recognition.
• Constant-latency inference: Regardless of how long the output sequence is, the KV cache remains constant at m+n, and inference latency and VRAM usage remain horizontally stable.

Technical Principles of Unlimited-OCR

• R-SWA attention mechanism: Each generated token attends only to all reference tokens (visual + prompt) and the most recent 128 output tokens. Visual tokens are excluded from the sliding-window state transition, preventing visual features from gradually blurring during long-range generation.
• Constant KV Cache design: The KV cache is implemented as a queue with capacity m+n. Each time a new token is generated, the (m+1)-th token is evicted, so computational cost and memory usage do not increase with sequence length.
• DeepEncoder visual encoding: Uses the SAM-ViT cascaded CLIP-ViT architecture, performs 16x token compression through the bridge layer, compresses a 1024×1024 image into 256 visual tokens, and freezes them after a single encoding.
• MoE-LLM decoder: A MoE architecture with 3B total parameters and 500M activated. All attention layers are replaced with R-SWA. It continues training for 4000 steps based on the DeepSeek-OCR checkpoint, with a global batch of 256 and a maximum sequence length of 32K.
• Inference engine optimization: Implements constant TPS and constant VRAM KV cache management in both Transformers and SGLang. Under the Flash Attention v3 kernel, per-call latency remains horizontal throughout.

Follow WeChat and reply "open source" to join theAI Open Source Project Exchange Group

How to Use Unlimited-OCR

• Model download: Get the code and weights via Hugging Face baidu/Unlimited-OCR or GitHub baidu/Unlimited-OCR .
• Environment preparation: Supports the Transformers library and the SGLang inference engine, and requires a corresponding GPU environment.
• Input format: Supports PDF page image input. Base mode is used for multi-page long documents, and Gundam mode is used for single-page high-resolution recognition.
• Inference call: A single forward pass can complete OCR transcription of an entire book/document, without the need for an external scheduler to process pages separately.
• Extended applications: The R-SWA mechanism can be transferred to long-output tasks such as ASR, translation, and subtitle generation.

Core Advantages of Unlimited-OCR

• SOTA recognition accuracy: OmniDocBench v1.5 overall score 93.23%, v1.6 overall score 93.92%, ranking first end-to-end.
• Constant resource usage: The KV cache does not grow with the number of document pages; memory usage and latency remain stable for long documents of 20 pages or 40+ pages.
• Speed advantage scales with length: The longer the output, the more pronounced the advantage; at 6144 tokens, the theoretical TPS upper limit leads DeepSeek-OCR by about 35%.
• General-purpose decoding architecture: R-SWA is not an OCR-exclusive trick; it applies to any generation task with a "reference source + long output".
• Lightweight and open source: 3B total parameters, 500M activated, with the model and code already open-sourced, facilitating deployment and secondary development.

Unlimited-OCR Project Address

• GitHub repository：https://github.com/baidu/Unlimited-OCR
• HuggingFace model library：https://github.com/baidu/Unlimited-OCR

Comparison of Unlimited-OCR with Similar Competing Products

Application Scenarios of Unlimited-OCR

• Enterprise archive digitization: Batch process hundreds or thousands of pages of scanned PDFs, ancient books, and bound volumes, enabling one-time structured extraction without splitting.
• Academic literature parsing: End-to-end transcription of entire papers, journal collections, and research reports, preserving formulas, tables, and reading order.
• Educational exam paper grading: Batch recognition of multi-page exam papers and workbooks, supporting color textbooks and complex layouts.
• Legal contract review: Precise OCR extraction of long contract texts for subsequent NLP analysis and compliance review.
• Multilingual translation pipeline: As a validation scenario for the general-purpose R-SWA decoding scheme, it can be extended to long-sequence tasks such as ASR and subtitle generation.