What is SmolDocling
SmolDocling (SmolDocling-256M-preview) is an efficient and lightweight multimodal document processing model. It can convert document images end-to-end into structured text, supporting the recognition of text, formulas, charts, and other elements, and is suitable for academic papers, technical reports, and many other types of documents. The model has only 256M parameters and features fast inference, processing each page in just 0.35 seconds on an A100 GPU. It is fully compatible with Docling and can export to multiple formats.
Main Features of SmolDocling
- Multimodal Document Conversion: It can efficiently convert image documents into structured text, supporting both scientific and non-scientific documents.
- Fast Inference: Processing one page of a document on an A100 GPU takes only 0.35 seconds and uses less than 500MB of VRAM.
- OCR and Layout Recognition: It supports optical character recognition (OCR) and preserves the document structure and bounding boxes of elements.
- Complex Element Recognition: It can recognize complex document elements such as code blocks, mathematical formulas, charts, and tables.
- Seamless Integration with Docling: It supports converting results into multiple formats (such as Markdown, HTML, etc.) and is compatible with Docling.
- Instruction Support: It supports multiple instructions, such as converting pages to Docling format, converting charts to tables, converting formulas to LaTeX, etc.
Technical Principles of SmolDocling
- Lightweight Design: SmolDocling-256M-preview is a vision-language model with only 256M parameters, designed specifically for document optical character recognition (OCR) and conversion. It can quickly process documents on consumer-grade GPUs, using less than 500MB of VRAM, with a processing time of only 0.35 seconds per document page.
- Visual Backbone Network: The model uses SigLIP base patch-16/512 as its visual backbone network, which has 93M parameters and can efficiently process image input. Through pixel compression technology, each 512×512 image patch is compressed into 64 visual tokens, significantly reducing computational resource requirements.
- Text Encoder: SmolDocling-256M-preview uses SmolLM-2 as its text encoder, which has 135M parameters and can process text input and fuse it with visual information.
- Multimodal Fusion and Output: The model can receive multimodal input of images and text and generate structured text output. It supports multiple document processing functions, including converting document images into structured text, extracting chart and table information, converting mathematical formulas into LaTeX format, etc.
- Optimized Dataset and Training Strategy: The training dataset of SmolDocling-256M-preview includes scientific and non-scientific documents, with document understanding accounting for 41%. During training, a higher pixel token rate (4096 pixels/token) was adopted, compared with the previous 1820 pixels/token, significantly improving efficiency.
SmolDocling Project Address
- HuggingFace Model Library：https://huggingface.co/ds4sd/SmolDocling-256M-preview
- arXiv Technical Paper：https://arxiv.org/pdf/2503.11576
Application Scenarios of SmolDocling
- Document Conversion and Digitization: SmolDocling-256M-preview can efficiently convert documents in image form into structured text format while preserving the document's original layout and complex elements (such as code blocks, mathematical formulas, charts, etc.). It supports multiple output formats, including Markdown, HTML, etc., and is suitable for document digitization processing.
- Scientific and Non-Scientific Document Processing: can handle non-scientific content (such as business documents, patent documents, etc.). Identify and extract key information from documents, such as formulas, charts, and tables.
- Fast OCR and layout recognition: SmolDocling-256M-preview provides efficient optical character recognition (OCR) capabilities, accurately extracting text from images while preserving document structure and element bounding boxes.
- Mobile and low-resource device support: SmolDocling-256M-preview can run on mobile devices or resource-constrained environments, such as smartphones or portable computers.