What is DeepDoc
DeepDoc is an open-source deep research tool focused on conducting deep research on local knowledge bases. Through a research-style workflow, the tool extracts text from local resources (such as PDF, DOCX, JPG, TXT, etc.), splits it, and stores it in a vector database for semantic similarity search. Users can generate a content structure based on instructions and provide feedback to optimize the structure. DeepDoc will generate a clear Markdown-format report. The tool is suitable for scenarios that require quickly extracting insights from local files, without the need to manually browse large numbers of files.
Main features of DeepDoc
- Local resource research: Supports multiple local file formats (PDF, DOCX, JPG, TXT, etc.), extracts and splits text content for subsequent processing.
- Semantic similarity search: Embeds text chunks into a vector database to achieve efficient semantic similarity search and quickly locate relevant content.
- Research-style workflow: Generates a content structure based on user instructions, supports feedback optimization, and improves the accuracy of research.
- Multi-step research process: Through steps such as generating knowledge, creating queries, and search optimization, it gradually generates high-quality report content.
- Structured report generation: Outputs clear Markdown-format reports for users to view and use.
Technical principles of DeepDoc
- Text extraction and splitting: Uses optical character recognition (OCR) technology to extract text from image files (such as JPG). The extracted text content is split into page-level chunks for subsequent processing.
- Vector database storage: Embeds the split text chunks into vector space and stores them in a vector database (such as Qdrant). This enables the system to efficiently perform semantic similarity search and quickly find the text chunks most relevant to the user's query.
- Multi-step research process: For each report section, the research agent generates knowledge and creates research queries. The search agent runs on local data to find the text chunks most relevant to the query. The reflection agent optimizes the search results to ensure that the generated content is accurate and useful. Finally, the content of each section is compiled into a complete report.
DeepDoc project address
- GitHub repository：https://github.com/Datalore-ai/deepdoc
Application scenarios of DeepDoc
- Academic research: Researchers can quickly organize and analyze large amounts of literature materials and generate structured research reports, saving time on manually organizing literature.
- Enterprise knowledge management: Enterprises can conduct in-depth mining of massive internal documents, reports, project materials, etc., quickly extract key information, and assist decision-making.
- Legal document analysis: Legal professionals can conduct in-depth analysis of large numbers of legal documents, cases, contracts, etc., quickly locate relevant clauses and cases, and improve work efficiency.
- Market research: Market researchers can conduct in-depth analysis of collected market research reports, consumer feedback, competitor materials, etc., and quickly generate market research reports.
- Personal knowledge management: Individual users can deeply organize and analyze personal notes, study materials, project documents, etc., quickly extract key information, and improve learning and work efficiency.