What is Dolphin
Dolphin is a lightweight, efficient document parsing large model open-sourced by ByteDance. Based on a two-stage method of parsing structure first and content second, the first stage generates a sequence of document layout elements, and the second stage uses the elements as anchors to parse content in parallel. Dolphin performs excellently on various document parsing tasks, surpassingGPT-4.1、Mistral-OCRand other models. Dolphin has 322M parameters, is small in size and fast in speed, and supports parsing of various document elements, including text, tables, formulas, etc. Dolphin's code and pretrained models have been made public, making it convenient for developers to use and research.
Dolphin's main features
- Layout analysis: Identifies various elements in a document (such as titles, charts, tables, footnotes, etc.) and generates a sequence of elements in natural reading order.
- Content extraction: Parses the entire document page into structured JSON format or Markdown format, facilitating subsequent processing and display.
- Text paragraph parsing: Accurately identifies and extracts textual content from documents, supporting multiple languages (such as Chinese and English).
- Formula recognition: Supports recognition of complex formulas, including inline formulas and block-level formulas, outputting LaTeX format.
- Table parsing: Supports parsing complex table structures, extracting cell contents, and generating tables in HTML format.
- Lightweight architecture: The model has 322M parameters, is small in size and fast in operation, and is suitable for use in resource-constrained environments.
- Supports multiple input formats: Supports processing multiple types of document images, including academic papers, business reports, technical documents, etc.
- Diverse output formats: Supports outputting parsing results in multiple formats such as JSON, Markdown, HTML, etc., making it convenient to integrate with different systems.
Dolphin's technical principles
- Page-level layout analysis: Uses Swin Transformer to encode the input document image and extract visual features. Based on the decoder, it generates a sequence of document elements, where each element contains its category (such as title, table, chart, etc.) and coordinate position. The goal of this stage is to generate structured layout information in natural reading order.
- Element-level content parsing: Based on the layout information generated in the first stage, crops a local view of each element from the original image. Uses specific prompts to perform parallel content parsing for each element. For example, tables use dedicated prompts to parse HTML format, and formulas and text paragraphs share prompts to parse LaTeX format. The decoder generates the final parsed content based on the cropped element images and prompts.
Dolphin's project address
- GitHub repository：https://github.com/bytedance/Dolphin
- HuggingFace model library：https://huggingface.co/ByteDance/Dolphin
- arXiv technical paper：https://arxiv.org/pdf/2505.14059
- Online experience Demo：http://115.190.42.15:8888/dolphin/
Dolphin's application scenarios
- Academic research: Parses text, formulas, and charts in papers, assisting literature organization and data analysis.
- Business office: Extracts key information from business documents, facilitating contract review and report generation.
- Education field: Digitizes textbooks and exam papers, supporting online learning and multilingual teaching.
- Technical development: Parses technical documents, facilitating code management and technical exchange.
- Daily applications: Quickly handle daily documents and improve office efficiency.