ocr image to markdown

OCR Image Recognition to Markdown

This skill allows you to "read" images and convert their content into editable Markdown text. This is especially useful when extracting data tables, slide content, or document screenshots, particularly when external OCR libraries are unavailable.

Usage Guide

1. Identify the target image:

   • Locate the image file you need to process.
   • If needed, use list_dir to browse the directory.

2. View the image:

   • Use the view_file tool to "see" the image content. The system allows you to directly process image data.
   • Key: You must use view_fileon the image path so that your visual model can digest it.

1. Transcribe the content:

   • Based on what you see, transcribe the text into Markdown.
   • Tables: Convert visually seen tables into standard Markdown tables (| 表头 | ... |)。
   • Headings: Use #, ## etc. to mark headings in the image, preserving the hierarchy.
   • Text: Transcribe paragraphs as plain text.
   • Numbers: Carefully verify all numbers, especially data in financial reports.

2. Save the output:

   • Use write_to_file to write the transcribed content into a .md file (for example ocr_results.md）。
   • If processing multiple images, consider appending them to the same file, or organize them logically.

Best Practice Tips

• Tables: Carefully align rows and columns. Standard Markdown tables do not support cell merging (rowspan/colspan). You need to expand merged cells according to the logical flow, or leave them blank.
• Complex layouts: If the image layout is complex (for example, left and right columns), serialize it in logical reading order (top to bottom, left to right).
• Charts/Graphics: If the image contains charts, describe the trends, or extract the visible data points as a list or table.
• No code execution required: Do not attempt to write or use Python libraries (such as pytesseract, easyocr, PIL) for text extraction. Please directly use your own visual capabilities.

Example Scenario

Request: "Convert these 3 screenshots of financial reports to markdown."

Execution:

1. list_dir View file: img1.png, img2.png, img3.png。
2. view_file Read img1.png。
3. (internal processing): Recognize the header "Q1 Revenue" and the table row data.
4. view_file Read img2.png and img3.png。
5. write_to_file Create financial_report.md and write the summarized content.