Mistral OCR 4 - Mistral AI's latest-generation document understanding model

What is Mistral OCR 4

Mistral OCR 4 is the latest-generation document understanding model released by Mistral AI. The model supports extracting text from complex documents such as PDFs, images, and presentations, and can return structured output with bounding-box localization, region type classification, and confidence scores. It supports 170 languages and is designed for downstream scenarios such as RAG, agentic workflows, and enterprise search.

Main features of Mistral OCR 4

• High-precision document parsing: Supports common enterprise formats such as PDF, DOC, PPT, and OpenDocument, extracting rich elements including text, tables, formulas, images, and signatures.
• Structured output: Each content block comes with bounding-box coordinates, a type label such as heading, table, formula, or signature, and per-page/per-word confidence scores.
• Multilingual support: Covers 170 languages across 10 language families, with particularly strong performance on low-resource languages.
• Dual-mode output: The same endpoint supports a pure extraction mode, Markdown + structured metadata, and Document AI mode (structured JSON output according to a custom Schema).
• Ultra-high throughput: A single node can process up to 2,000 pages of documents per minute, making it suitable for large-scale batch processing scenarios.

Technical principles of Mistral OCR 4

• Visual text detection and classification: Based on CNN/Transformer architectures, it locates text regions in documents and performs semantic classification on each block, restoring the hierarchical structure of the original document.
• Sequence-to-sequence text recognition: Uses Seq2Seq or CTC models to convert detected character streams into editable text, improving recognition accuracy in combination with image preprocessing.
• Structured semantic chunking: Splits documents into semantic chunks with type labels and coordinates, providing downstream RAG systems with citation-ready units that can be used directly for retrieval and supporting structured operations by agents.
• Lightweight single-container deployment: The model is compact enough to be self-hosted and deployed within a single container, meeting data residency and compliance requirements.

How to use Mistral OCR 4

• Register to get an API Key: Visit Mistral AI's developer platform La Plateforme, register an account, then go to the "API Keys" page to create and save a key.
• Install the SDK: In a Python environment, run pip install mistralai to install the official SDK.
• Call the API to process documents Use the client.ocr.process() method, pass in a document URL or local file, set model="mistral-ocr-latest", and enable include_blocks=True to get bounding boxes and type information. You can also use confidence_scores_granularity="word" to get per-word confidence scores.
• Parse the structured results: The API returns JSON containing a pages array. Each page includes fields such as markdown text,images、tables、hyperlinks、dimensions , and confidence_scores , and can be directly integrated into RAG or agentic workflows.
• Batch processing to reduce costs: For high-throughput scenarios, it is recommended to submit tasks through the Batch Inference API to enjoy a 50% price discount.