SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsmarkdown-conversion

markdown-conversion

Office & productivity

View repository

Updated 10-15 22:15

About this skill

Markdown Conversion

Convert any supported source—file, directory, or URL—to clean Markdown. Each input generates one .md file (by default, images are also extracted to <stem>_files/）。

Standardized Output Contract

The output of this skill is a "readable Markdown layer" intended for downstream processes such as PPT / Word / study notes, rather than a reproduction of the original file's layout. All converters follow the same set of conventions:

Output Convention
Main Markdown Each input generates one .md; headings, paragraphs, lists, tables, and links should be kept readable and citable first
Asset directory Companion resources such as images are placed in a sibling <stem>_files/, and only relative paths are used in Markdown
Image manifest Converters that can provide structured image information will write image metadata to <stem>_files/image_manifest.json ; including source type, dimensions, usage count, page / slide occurrence locations, etc.
Conversion profile After a successful conversion, write <stem>.conversion_profile.json, recording the original file, Markdown, asset directory, manifest, title / table / image / link counts, and converter information; during batch conversion, each Markdown has its own independent sidecar to avoid overwriting each other
Web image sources web_to_md.py When downloading web images, write <stem>_files/image_sources.json; the image licensing status defaults to unknown, and must be reviewed before delivery
Tabular data Excel, PPT tables, and readable chart data are uniformly converted to Markdown tables, avoiding data remaining only in images or placeholders
Links External links are preserved as Markdown links whenever possible; unsafe or unparseable links are downgraded to plain text / placeholder descriptions
Original files This skill does not move original files; project-based workflows such as PPT Master / Word Master should archive the original files, Markdown, and asset directory themselves after conversion

Format boundaries:

  • The Markdown layer prioritizes "content understanding" and "regeneration"; it does not promise to preserve original margins, animations, themes, or layout coordinates.
  • Office vector resources (such as EMF / WMF), if extracted by the backend, should be retained as assets; they should not be rasterized to PNG by default.
  • The copyright / licensing status of images downloaded from the web is not determined by the converter; the source must be reviewed separately before formal delivery.

Design philosophy: two orthogonal dimensions

维度 A：还原力度
  默认       = 自然转换（应用合理的清理：网页正文识别、PDF 页眉页脚去重、字幕段落锚点）
  --raw      = 完美还原（关闭所有启发式清理，保留原始结构）

维度 B：图片处理（互斥三选一）
  默认           = 全部保留
  --filter-images = 按尺寸和宽高比过滤装饰小图；保留重复出现位置，不以压缩率判断正文图
  --no-images    = 完全不抽图，Markdown 里也不留 ![](...) 引用

The two dimensions can be freely combined.No preset bundles—each flag handles only one thing, and combinations can be layered.

Quick Start

The unified dispatcher automatically identifies the input type and can accept one or more files / URLs / directories at a time:

python3 scripts/convert.py <文件或URL> [<文件或URL> ...]

Default output:<输入目录>/<文件名>.md. For a local single file, you can use -o <output.md> to specify the output path; for multiple inputs or directory inputs,-o is the output directory. After success, the dispatcher will print OUTPUT: /绝对路径/output.md。

# 默认：自然转换 + 图全保留（最常用）
python3 scripts/convert.py paper.pdf                       # PDF（本地 PyMuPDF）
python3 scripts/convert.py paper.pdf --mineru              # PDF（MinerU 云端 OCR）
python3 scripts/convert.py report.docx                     # Word
python3 scripts/convert.py data.xlsx                       # Excel
python3 scripts/convert.py deck.pptx                       # PowerPoint
python3 scripts/convert.py book.epub                       # EPUB
python3 scripts/convert.py https://example.com/post        # 网页（trafilatura 正文识别）
python3 scripts/convert.py paper.pdf report.docx deck.pptx # 多来源批量
python3 scripts/convert.py ./course_dir -t sub             # 字幕批量（含段落锚点）
python3 scripts/convert.py ./mixed_docs                    # 目录批量
python3 scripts/convert.py report.docx --json              # 额外打印机器可读 JSON 结果

# 完美还原（关闭启发式清理）
python3 scripts/convert.py paper.pdf --raw                 # 保留页眉页脚，不识别标题层级
python3 scripts/convert.py https://example.com --raw       # 保留导航/侧栏/footer
python3 scripts/convert.py lecture.srt --raw               # 字幕回到单行拼接

# 图片处理
python3 scripts/convert.py paper.pdf --filter-images       # 过滤装饰图（保留信息图）
python3 scripts/convert.py paper.pdf --no-images           # 纯文本

# 组合
python3 scripts/convert.py paper.pdf --raw --no-images     # 完美还原 + 纯文本

What "cleanup" specifically means for each backend

Backend Default app cleanup (--raw Disabled)
pdf_to_md.py(Local) Header/footer deduplication, font size → heading level recognition; low-confidence table misdetections fall back to body text; use --render-vector-figures to explicitly render large vector graphics as PNG
pdf_to_md_mineru.py MinerU cloud processing, no local cleanup that can be disabled (--raw is a no-op); image filtering is performed locally after the result is downloaded
doc_to_md.py（docx / html / epub / ipynb / pandoc） No explicit cleanup (mammoth/nbconvert/ebooklib are already faithful conversions; the html path removes <head>/<style>/<script>, treated as necessary rather than heuristic); DOCX preserves text tables as pipe Markdown, converts OMML / Office Math formulas to LaTeX in place, and preserves EMF / WMF assets
ppt_to_md.py None (python-pptx reads directly, no cleanup; preserves soft line breaks, field text, real list numbering, tables, chart data, SmartArt nodes, external / internal hyperlinks, and image manifest)
web_to_md.py trafilatura body text recognition; falls back to upstream body text recognition when unavailable or no result; raw preserves the full HTML body
subtitle_to_md.py Paragraph chunking + per 50 <!-- Block N --> <!-- HH:MM:SS --> anchors

Batch directory conversion

convert.py accepts a directory argument and converts all supported files in it (one level deep). Each file is processed by the corresponding converter, output as <文件名>.md(or written to -o the specified directory).

python3 scripts/convert.py ./mixed_docs               # 原位转换每个文件
python3 scripts/convert.py ./mixed_docs -o ./out      # 所有 .md 写入 ./out/

Batch mode continues after a single file fails, and finally prints success / failure / skipped counts.

Very large PDFs (books/long reports) can first be split with pdftk、qpdf or PyPDF2 and then converted—the converter also warns when a single PDF exceeds about 200 pages.

Supported sources

Type Extension / input Converter
PDF (text-based) .pdf pdf_to_md.py（PyMuPDF）
PDF (scanned, formula-dense, complex layout) .pdf + --mineru pdf_to_md_mineru.py
Word / EPUB / HTML / Jupyter .docx .epub .html .htm .ipynb doc_to_md.py(native)
Other office / academic formats .doc .odt .rtf .tex .rst .org .typ doc_to_md.py(pandoc fallback)
Spreadsheets .xlsx .xlsm excel_to_md.py
Slides .pptx .pptm .ppsx .ppsm .potx .potm ppt_to_md.py
Subtitles .srt .vtt .ass(single file, flat directory, or course directory) subtitle_to_md.py
Web pages http:// / https:// web_to_md.py（Python，curl_cffi）
Plain text .txt pass-through
Already Markdown .md .markdown pass-through

.xls and legacy .ppt are not parsed directly—please first save as .xlsx / .pptx。.doc handled via pandoc fallback.

Calling converters directly

convert.py is the recommended entry point, but each backend can also be used as a standalone CLI:

python3 scripts/pdf_to_md.py book.pdf                         # 默认自然转换
python3 scripts/pdf_to_md.py book.pdf --filter-images         # 过滤装饰图
python3 scripts/pdf_to_md.py book.pdf --raw                   # 完美还原
python3 scripts/pdf_to_md.py book.pdf --render-vector-figures # 显式渲染矢量图区域
python3 scripts/pdf_to_md_mineru.py scan.pdf                  # 需要 MINERU_API_TOKEN
python3 scripts/pdf_to_md_mineru.py scan.pdf --no-images
python3 scripts/doc_to_md.py paper.tex                        # 使用 pandoc
python3 scripts/doc_to_md.py report.docx --filter-images
python3 scripts/excel_to_md.py report.xlsm --max-rows 200 --max-cols 40
python3 scripts/convert.py data.xlsx --include-hidden        # 包含隐藏工作表
python3 scripts/ppt_to_md.py deck.pptx --filter-images        # 过滤母版背景/装饰
python3 scripts/web_to_md.py https://example.com              # 默认 trafilatura 正文识别
python3 scripts/web_to_md.py https://example.com --raw        # 关闭正文识别（保留 nav/footer）
python3 scripts/subtitle_to_md.py lecture.srt                 # 默认段落 + Block 锚点
python3 scripts/subtitle_to_md.py lecture.srt --raw           # 单行拼接（旧行为）

Each script outputs <输入>.md and the <输入>_files/of embedded images, referenced in Markdown using relative paths. Backends that support image manifests additionally write <输入>_files/image_manifest.json, used downstream to determine image dimensions, source, and occurrence position. A successful conversion writes <stem>.conversion_profile.json; the unified scheduler also supports --json to print machine-readable results. All image-related backends support --no-images and --filter-images(mutually exclusive); heuristic cleanup can be disabled with --raw .

Excel values and structure

  • By default, visible worksheets are exported, and the names and hints of hidden / very hidden worksheets are listed; --include-hidden exports these worksheets and indicates their status.
  • Plain numeric values retain Excel's 15 significant digits, avoiding rounding to the old 6 significant digits; percentages are multiplied by 100 exactly and have % appended, without truncating to the displayed number of decimal places.
  • Integer identifier formats (such as 0000) retain leading zeros; date-times retain fractional seconds. Other unknown formats with units or scaling retain the original value, and warnings are summarized by worksheet.
  • Merged cells inherit the anchor value and format; numeric merged ranges are annotated as shared values to avoid duplicate values being mistaken for independent data that can be summed; plain text merges do not add this annotation.
  • Formulas read the cached results from the file and do not recalculate the formula; --max-rows / --max-cols still control the output limit.

Select the PDF backend

Always use the local parser first, and check the output before deciding whether to switch.

Case Action
Output is readable Done—keep the local result
Garbled text, disordered reading order, missing content Switch to --mineru and run again
Scanned documents, image-only PDFs Use --mineru
PDFs from URLs convert.py automatically routes to MinerU

MinerU requires MINERU_API_TOKEN, or copy resources/config.example.json to the gitignored resources/config.json and fill in the token.

Web scraping

web_to_md.py supports all URLs. After installing curl_cffi , it can emulate a Chrome TLS fingerprint, enabling scraping of WeChat official accounts (mp.weixin.qq.com) and other sites that block Python's default fingerprint—no extra parameters required. By default it verifies certificates and rejects private network targets; trusted intranet pages can explicitly use --allow-private-hosts, and self-signed certificates can use --insecure. When not installed, it falls back to standard requests(sufficient for most public websites).

Environment diagnostics

python3 scripts/check_env.py    # 按格式显示就绪状态：Python 依赖、pandoc、MinerU token

Regression tests

Offline regression tests use synthetic input, and MinerU downloads are simulated by a local ZIP without accessing the network (requires Pillow and requests):

cd skills/markdown-conversion
python3 -m unittest discover -s tests

When a MinerU ZIP contains multiple Markdown files, the main output uses the first one; the remaining files are kept in the original extraction directory, and their paths are listed via a stderr warning for checking fragments or alternative versions.

Installation

pip install -r resources/requirements.txt
python3 scripts/check_env.py

check_env.py prints a readiness table categorized by format—green means available, and missing dependencies indicate the required package or binary.

pandoc optional—only needed when processing long-tail document formats (.doc / .odt / .rtf / .tex / .rst / .org / .typ).

trafilatura recommended—web_to_md.py uses it by default for main content detection (stripping navigation/ads/sidebars/comments). If not installed, it falls back to the built-in heuristic and prints a one-line notice;--raw mode does not use it.

Troubleshooting

Symptom Solution
When converting .doc/.tex , etc., it reports pandoc not found brew install pandoc(macOS) or sudo apt install pandoc
MinerU call reports an authentication error Set MINERU_API_TOKEN, or copy resources/config.example.json to the gitignored resources/config.json and fill in the token
WeChat / Cloudflare URLs return 403 Install curl_cffi to let web_to_md.py emulate a real Chrome TLS fingerprint
Automatic type detection is wrong Use -t pdf|doc|excel|pptx|web|sub to force it
File extension is abnormal (such as .pdf.bak） uses -t Force specified type

Output conventions

  • Input file → <输入目录>/<文件名>.md(unless specified -o）
  • Embedded images → <输入目录>/<文件名>_files/, referenced in Markdown using relative paths
  • Image manifest → <输入目录>/<文件名>_files/image_manifest.json(generated when supported by the backend and images are present)
  • Web image sources → <输入目录>/<文件名>_files/image_sources.json(generated when the web backend downloads images, authorization status defaults to unknown）
  • Conversion profile → <stem>.conversion_profile.json(records the artifacts and structural statistics of this conversion)
  • URL → the current project's projects/(unless specified -o); PDF URLs processed via MinerU use MinerU's output directory behavior
  • Input that is already Markdown / plain text is output (or copied) directly without conversion