What is Docmatix
Docmatix is a large-scale dataset designed for the Document Visual Question Answering (DocVQA) task. It contains 2.4 million images and 9.5 million question-answer pairs, with data sourced from 1.3 million PDF documents. The scale of the Docmatix dataset is 240 times that of previous datasets, providing abundant resources for training and optimizing vision-language models (VLMs).
Main Features of Docmatix
- Large-Scale Data Coverage: Docmatix contains 2.4 million images and 9.5 million question-answer pairs, with data sourced from 1.3 million PDF documents, providing abundant resources for training and evaluating vision-language models.
- Diverse Document Content: The dataset covers various types of documents, including scanned images, PDF files, and digital documents, and the documents contain both text and visual features.
- High-Quality Question-Answer Pairs: Through automated tools and manual review, the quality and accuracy of the question-answer pairs are ensured.
- Supports Model Training and Fine-Tuning: Docmatix is used to train and fine-tune vision-language models, improving their performance in understanding and answering questions related to document content.
Technical Principles of Docmatix
- Data Sources and OCR Processing: The Docmatix dataset is generated based on the PDFA dataset, which contains 2.1 million PDF documents. After optical character recognition (OCR) processing, the text in images is converted into machine-readable text data.
- Automatic Question-Answer Pair Generation: Based on the Phi-3-small model, question-answer pairs are automatically generated from OCR-transcribed text. The entire process is automated, aiming to create a large number of question-answer pairs related to document content.
- Data Cleaning and Filtering: The creators of Docmatix filtered the question-answer pairs generated by the model and discarded those identified as inaccurate or irrelevant.
- Dataset Construction: When constructing the dataset, each row corresponds to a PDF file and contains the image path and related question-answer pairs. The original PDFs of all samples can be traced back to the PDFA dataset, providing transparency and reliability.
Docmatix Project Address
- GitHub Repository：https://github.com/huggingface/docmatix
- HuggingFace Model Library：https://huggingface.co/datasets/HuggingFaceM4/Docmatix
How to Use Docmatix
- Visit Hugging Face Hub: Go to Hugging Face Hub to download the dataset.
- Load the Dataset: Use Hugging Face's
datasetslibrary to load the dataset. - Explore the Data: View samples in the dataset to understand its structure and content.
- Fine-Tune the Model: Use the dataset to fine-tune language models, such as Florence-2.
- Evaluate Performance: Evaluate model performance on the validation set to ensure it meets the expected goals.
Application Scenarios of Docmatix
- Automated Customer Service: Models trained with Docmatix are used in automated customer service systems to understand and answer questions about product manuals, terms of service, or FAQ documents.
- Intelligent Document Analysis: In the legal, financial, or medical fields, intelligent document analysis can help professionals quickly extract key information from large numbers of documents, such as extracting clauses from contracts or diagnostic information from medical records.
- Education and Academic Research: In the field of education, Docmatix helps develop assisted learning tools, such as automatically generating questions and answers, to help students better understand course materials. In academic research, it is used to automate the literature review process.
- Business Process Automation: In enterprises, it automates the processing of invoices, reports, application forms, and other documents, greatly improving efficiency and reducing manual intervention.
- Information Retrieval Systems: Docmatix helps develop more advanced information retrieval systems that can understand users' questions and retrieve from large numbers of documents.