What is olmOCR
olmOCR is an open-source tool released by Ai2 for efficiently converting PDF documents into clean, structured plain text. olmOCR combines document-anchoring technology with Qwen2-VL-7B-Instruct (Alibaba's multimodal model), and supports processing many types of PDF documents, including academic papers, books, tables, and charts. olmOCR works by extracting text and layout information from documents and combining it with page images to extract content more accurately and preserve structured information. olmOCR supports large-scale batch processing, with a processing cost of only $190 per million pages, far lower than other commercial solutions.
Main Features of olmOCR
- Efficient document conversion: Converts PDF documents into clean, structured plain text while preserving the structured content of the document (such as chapters, tables, lists, formulas, etc.).
- Supports multiple document types: Processes PDF documents from different domains, including academic papers, legal documents, brochures, charts, and scanned documents, etc.
- Document-anchoring technology: Based on extracting the positional information of text blocks and images in the document, it combines this with the original text to form a prompt, improving the accuracy of content extraction.
- Large-scale processing capability: Optimizes the inference workflow and supports batch processing from a single document to millions of pages, with extremely low cost (only $190 per million pages).
- Open source and extensibility: All components (including model weights, data, and training code) are open source and support multiple inference frameworks (such as vLLM and SGLang), making it easy for users to extend and customize.
Technical Principles of olmOCR
- Document-anchoring: Based on extracting the positional information of text blocks and images in PDF pages, it combines this with the original text to form a prompt. The prompt, together with the rasterized image of the page, is input into a vision-language model (VLM). This helps the model better understand the structure and layout of the document and reduces extraction errors caused by blurry images or complex layouts.
- Fine-tuned vision-language model (VLM): A 7B-parameter vision-language model based on Qwen2-VL-7B-Instruct. It is fine-tuned on a dataset containing 260,000 pages of PDFs to adapt to document processing tasks. The model outputs structured JSON data, including page metadata (such as language, orientation, whether tables are included, etc.) and text content in natural reading order.
- Efficient inference and cost optimization: Uses efficient inference frameworks such as SGLang and vLLM to support large-scale parallel processing. Based on optimized hardware utilization and inference workflow, olmOCR's processing cost is extremely low, only $190 per million pages, far lower than other commercial solutions.
- Enhanced robustness: When extraction fails or repeated generation occurs, it automatically retries and adjusts the prompt content. It automatically detects page orientation and performs rotation correction to ensure correct content extraction.
olmOCR Project Address
- Project official website：https://olmocr.allenai.org/
- GitHub repository：https://github.com/allenai/olmocr
- HuggingFace model library：https://huggingface.co/collections/allenai/olmocr
- Technical paper：https://olmocr.allenai.org/papers/olmocr
Application Scenarios of olmOCR
- Language model training: Extracts high-quality text from PDF documents to provide training corpora for language models.
- Academic research: Quickly converts academic papers into structured text, supporting literature reviews and knowledge mining.
- Legal document processing: Accurately extract legal documents and contract content, supporting legal text analysis and compliance checks.
- Enterprise document management: Convert internal enterprise PDF documents into editable text for easy management and updates.
- Digital libraries and archive digitization: Convert PDF scans of paper books and historical documents into electronic documents to achieve digital preservation and dissemination.