Dolphin is the official open-source repository for "Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting," a paper accepted at ACL 2025. The project focuses on the document image parsing task, with the goal of addressing the challenges posed by diverse document types and the complex interweaving of elements such as text paragraphs, charts, formulas, tables, and code blocks.
In terms of use, Dolphin is used to parse document images into structured results, supporting page-level parsing (parsing an entire page document into structured JSON and Markdown formats) and element-level parsing (parsing individual document elements such as text, tables, and formulas), while also providing layout parsing capabilities.
In terms of main capabilities, Dolphin-v2 is an enhanced general document parsing model that can handle any document type, whether digitally native or photographed. It adopts a document-type-aware two-stage architecture with extensible anchor prompting: the first stage performs document type classification (digital documents and photographed documents) as well as layout analysis and reading order prediction; the second stage adopts a hybrid parsing strategy, performing holistic parsing for photographed documents and parallel element-by-element parsing for digital documents. Dolphin-v2 has been upgraded to 3B parameters, supports 21 categories of element detection, attribute field extraction, dedicated formula and code parsing, and enhances the robustness of photographed document parsing. Dolphin-1.5 improves parsing performance while maintaining a lightweight 0.3B architecture. The project also features element sequence generation in natural reading order, heterogeneous anchor prompting for different document elements, an efficient parallel parsing mechanism, and support for Hugging Face Transformers for integration.
In terms of usage, the installation steps are: clone the repository (git clone https://github.com/ByteDance/Dolphin.git，cd Dolphin), install dependencies (pip install -r requirements.txt), download the Dolphin-v2 pretrained model (you can visit the Hugging Face model card, or use git lfs install followed by git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model, or install huggingface_hub via pip install huggingface_hub and then run huggingface-cli download ByteDance/Dolphin-v2 --local-dir ./hf_model). The project provides two inference frameworks and supports two parsing granularities: page-level parsing and element-level parsing.
In terms of applicable scenarios, it is suitable for document parsing tasks that require extracting structured information from digitally native or photographed documents, including layout analysis, reading order restoration, parsing of elements such as tables/formulas/code, and parsing of multi-page PDF documents.
In terms of limitations, the materials do not provide more details on fees, pricing, platform support scope, specific performance figures, or usage steps, and these are not currently available. The project welcomes users to submit cases where the model performs poorly to issues so that it can be continuously optimized.
