What is Nanonets-OCR-s
Nanonets-OCR-s (Nanonets OCR Small) is an image-to-Markdown OCR model launched byNanonetsIt supports converting document content in images into a structured Markdown format. The model can extract text and supports intelligent recognition and processing of complex document elements, such as LaTeX equations, image descriptions, signatures, watermarks, checkboxes, and complex tables. Nanonets-OCR-s is based on a deep learning model and has been trained on a large amount of data. It supports a variety of document types, including research papers, financial documents, and medical forms. The output Markdown-format content can be directly processed by large language models and is widely used in academic, legal, financial, and enterprise fields, greatly improving the efficiency and accuracy of document processing.
Main features of Nanonets-OCR-s
- LaTeX equation recognition: Automatically converts mathematical equations and formulas into correctly formatted LaTeX syntax, including the conversion of inline mathematical expressions and display equations.
- Intelligent image description: Uses structured tags to describe images in documents so that they can be processed by large language models. Supports describing the content, style, and context of single or multiple images (such as logos, charts, graphs, QR codes, etc.), and predicts image descriptions in <img> tags, with page numbers predicted in <page_number> tags.
- Signature detection and isolation: Identifies and isolates signatures in documents, which is crucial for legal and business document processing. The model predicts signature text in <signature> tags.
- Watermark extraction: Similar to signature detection, the model supports detecting and extracting watermark text from documents, and the predicted watermark text is located in <watermark> tags.
- Intelligent checkbox processing: Converts checkboxes and radio buttons in forms into standardized Unicode symbols to achieve consistent processing. The model predicts the status of checkboxes in <checkbox> tags.
- Complex table extraction: Extracts complex tables from documents and converts them into Markdown and HTML tables.
Technical principles of Nanonets-OCR-s
- Vision-language model (VLM): Nanonets-OCR-s is based on a vision-language model (VLM). The model simultaneously understands and processes visual information (such as images, tables, charts, etc.) and language information (such as text content). The model is based on jointly learning visual and language features to better understand the structure and content of documents.
- Dataset curation and training: To train the model, a document dataset containing more than 250,000 pages was curated, covering a variety of document types, such as research papers, financial documents, legal documents, medical documents, tax forms, receipts, and invoices. The documents contain elements such as images, charts, equations, signatures, watermarks, checkboxes, and complex tables. Synthetic datasets and manually annotated datasets were used for training. First, the model was trained on the synthetic dataset, and then fine-tuned on the manually annotated dataset. The synthetic dataset provides a large number of training samples, and the manually annotated dataset can improve the model's performance on real documents.
- Base model selection: The Qwen2.5-VL-3B model was selected as the base model for the vision-language model (VLM) and fine-tuned on the curated dataset to improve its performance on document-specific optical character recognition (OCR) tasks.
- Intelligent content recognition and semantic tagging: Nanonets-OCR-s can recognize various elements in documents and semantically tag them. Based on this approach, the model converts unstructured document content into a structured, context-rich Markdown format, providing higher-quality input for downstream tasks.
- Model Optimization and Adjustment: During the training process, the model's parameters and structure are continuously optimized to improve performance across various document types and scenarios. Specific adjustments and optimizations are made to the model for different functional requirements to ensure its accuracy and reliability in practical applications.
Nanonets-OCR-s Project Address
- Project Official Website：https://nanonets.com/research/nanonets-ocr-s/
- HuggingFace Model Repository：https://huggingface.co/nanonets/Nanonets-OCR-s
Application Scenarios of Nanonets-OCR-s
- Paper Digitization: Convert academic papers containing LaTeX equations and tables into a structured Markdown format, making it convenient for researchers to organize literature, cite, and conduct further analysis.
- Research Material Organization: Quickly extract key information from research papers, such as experimental data, charts, and conclusions, facilitating researchers in quick reference and comparison.
- Academic Publishing: Help publishers convert academic literature in paper or PDF format into formats suitable for online publication, improving the accessibility and searchability of the literature.
- Legal Document Analysis: Quickly identify and extract important clauses, case citations, and legal provisions from legal documents, improving the efficiency of legal research and case analysis.
- Financial Statement Processing: Extract data from financial statements, such as revenue, expenses, and balance sheets, facilitating financial analysis and report generation.