pdf-processor

pdf-processor

Positioning

This skill is the unified entry point for PDF processing, covering scanned document preprocessing, OCR dual-layer PDF generation, page number addition, PDF merging, decryption, watermark removal and compression. Prioritize protecting original files, and choose the shortest usable workflow according to user intent.

Core responsibilities:

1. One-click scanned document processing: decryption → page preprocessing → merge output → OCR dual-layer PDF.
2. Single-item processing: only preprocessing, only OCR, only compression, only decryption, only watermark removal, only merging, only page number addition.
3. When the user does not specify otherwise, scanned documents go through the default unified entry point; when a single-item need is explicitly stated, only the corresponding tool is executed.

This skill does not perform plain-text PDF content editing, PDF reading annotation, electronic signatures, or format conversion not for compression purposes.

Default strategy

• Do not modify original files; output to a new file, and when names conflict, add _1、_2 and other sequence numbers.
• Scanned documents, photographed documents and evidentiary materials continue by default to generate searchable dual-layer PDFs; PaddleOCR and local ocrmypdf by default directly retain the original PDF, and only MinerU or explicit image processing requests go through unified raster preprocessing.
• auto When configured, by default prioritize the PaddleOCR API, then try MinerU, and finally fall back to the local engine (when RapidOCR is installed, prioritize RapidOCR, otherwise ocrmypdf); materials explicitly prohibited from external transmission must use --local-only。
• When the input contains sensitive information such as names, case numbers, medical information, account numbers, etc., and the user has not yet explicitly authorized external transmission of the file, first explain that the complete PDF will be uploaded and obtain confirmation once; after the current file has been authorized, do not ask repeatedly, and the authorization does not extend to the same directory or other materials.
• Electronic PDFs or mixed PDFs by default retain text, vector, image and annotation layers, skipping rasterization preprocessing and recompression; only when the user explicitly accepts the risk of layer loss use --force-raster-preprocess。
• "only preprocessing", "no OCR", "only correction and compression" only then use --preprocess-only。
• "merge", "add page numbers", "decrypt", "remove watermark", "compress" only execute the corresponding tool, and do not automatically enter preprocessing/OCR.
• Compression is performed separately only when the user explicitly requests it; the default compression in the unified entry point is part of the preprocessing output strategy.
• Watermark removal is performed only when the user explicitly requests it, and is not a default automatic step.

Common workflows

1. One-click processing of scanned PDFs

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf

auto When the configured PaddleOCR is selected, by default the original PDF is sent directly to PP-OCRv6, skipping unified rasterization and pre-OCR compression, in order to preserve scanning resolution, image layers and API coordinate space; when the actual backend is determined to be local ocrmypdf the original scanned pages are likewise retained, and OCRmyPDF's built-in orientation detection, deskewing and cleanup are used. The MinerU or explicit preprocessing path still targets medium 200 DPI, JPEG quality 72, chroma subsampling 1, and uses 25MP to protect abnormally large canvases. When cropping or unified re-rasterization is truly needed, use --enable-crop or --force-raster-preprocess. Electronic/mixed PDFs automatically retain the original layers. When file size limits are very strict, use:

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf --compress-level high

When it is truly necessary to retain extremely large rasters, the upper limit can be explicitly raised;--max-preprocess-megapixels 0 will disable protection, but may significantly increase memory usage and the risk of OCR skipped pages.

Sensitive materials use a unified entry point but must not be shared externally:

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf --local-only

Large batches of scans with the correct page orientation can be sped up:

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf \
  --skip-coarse-rotation --preprocess-jobs 6 --preprocess-chunk-pages 80

2. Preprocessing only, no OCR

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf --preprocess-only

Only perform page correction, no compression, no OCR:

python3 scripts/pdf-preprocess-ocr.py --input input.pdf --output output.pdf \
  --preprocess-only --no-compress

3. OCR text layer only

python3 scripts/pdf-ocr.py --input input.pdf --output output.pdf

The default backend is auto: when PaddleOCR is configured, first use PP-OCRv6 line-level coordinates to generate the text layer, then try MinerU according to OCR_API_ORDER ; when the external service fails or is not configured, fall back to the local engine—when RapidOCR is installed, prefer local RapidOCR (good Chinese line-level recognition quality, does not leave the machine), otherwise fall back to local ocrmypdf. This default path uploads the complete PDF; when external transmission is not allowed, use:

python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf --local-only

--local-only also follows the local-first chain (RapidOCR → ocrmypdf). For local Chinese OCR of sensitive materials, it is recommended to install RapidOCR first:

pip install rapidocr

--allow-external-upload is only a compatibility parameter for old commands and no longer controls backend selection. Paddle's server-side orientation correction and dewarping are disabled by default to avoid inconsistency between OCR coordinates and the original image space.

# 强制本地兜底
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf --backend local_ocrmypdf

# 强制本地 RapidOCR（onnx 本地推理，中文质量好且不出本机）
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf --backend rapidocr_local

# 强制 PaddleOCR API；PP-OCRv6 是双层 PDF 默认模型
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf \
  --backend paddle_api --paddle-model PP-OCRv6

# 干净扫描件/表格可试 PP-StructureV3 的 overall_ocr_res 行级坐标
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf \
  --backend paddle_api --paddle-model PP-StructureV3

# 强制 MinerU API
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf --backend mineru_api

# 显式保存 OCR 可读文本和运行元数据；默认不归档案件材料
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf --archive-results

For backend selection, API configuration, and protocol details, see references/ocr-backend-guide.md、references/paddleocr-api-guide.md、references/mineru-api-guide.md。

PaddleOCR-VL-1.5/1.6 can also parse, but only provides block-level coordinates, and the text layer positioning granularity is lower than PP-OCRv5/v6 / PP-StructureV3. This skill does not integrate visual recognition pipelines such as Qwen/GLM, nor does it claim that cloud results contain character-level coordinates.

The PP-OCRv6 text model enables --actualtextby default: after the text layer is generated, call PP-StructureV3 once more to obtain the layout, fuse natural paragraphs, and write them into the PDF with /ActualText marked-content. The Paddle text layer also uses local geometric rules to identify multi-line body paragraphs, uniformly reducing the font size within a paragraph down to the existing minimum safe value, while preserving the original line coordinates and horizontal box width, to improve paragraph inference in PDF Expert.--no-actualtext can be disabled;--layout-dump FILE reuse an existing layout dump to avoid a second API call. For reader compatibility, see Section 4 below.

4. Natural paragraph text and PDF copy line breaks

The text layer of a double-layer PDF is stacked line by line (to protect selection coordinate precision), so copying directly from the PDF will break lines according to physical lines. There are two solutions:

Default:/ActualText + standalone Markdown. --actualtext(enabled by default) writes natural paragraphs into the PDF's /ActualText marked-content, while preserving line-level glyph coordinates. Actual support in various readers:

Extraction method	Paragraph continuity when copying from PDF
Poppler（pdftotext -raw、pdftotext default)	✅ Whole paragraph continuous, no line breaks
PDF Expert	⚠️ Ignores /ActualText, but infers paragraphs by itself based on font size and geometry; normalizing Paddle body paragraph styles can reduce line breaks, but complete elimination is not guaranteed
PyMuPDF, pypdf, macOS Preview (PDFKit)	❌ Still breaks lines according to physical lines

Therefore Poppler users can obtain paragraph-level text directly from the PDF; PDF Expert can only be improved through reader heuristics; macOS Preview users will still get line breaks when copying. When stable paragraph text is needed across readers, explicitly output standalone Markdown; by default, no body-text copy is generated next to case materials:

python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf \
  --backend paddle_api --clean-text-output output-clean.md

Standalone Markdown can also be generated by pdf_ocr_paragraphs.py . If an individual natural paragraph skips excluded lines (such as seal fragments), it cannot be wrapped because the physical lines are discontinuous, and it will degrade to line level (text is not lost; only copying that paragraph breaks by line).

When it is necessary to resolve intra-paragraph carriage returns, extra blank lines, and seal text in copied text, use a division of labor between two models:PP-OCRv6 provides the main line-level text and coordinates,PP-StructureV3 provides reading order, region types, and seal region. Only when the v6 line confidence is below 0.80, the corresponding Structure line confidence is not below 0.90, the coordinates of both sides overlap heavily, and the latter is at least 0.10 higher, is the Structure line-level text fallback used; layout block text is never read, and no large language model is invoked.

Structure block boundaries serve only as candidates rather than mandatory segment boundaries: the fuser first merges fragments of the same visual line, within the text region restores physical line breaks according to line spacing, indentation, and right boundary, then connects body text across adjacent text blocks based on sentence-ending punctuation and clause numbering;table the region preserves cell order by visual line and uses | as the separator. This handles pages where Structure has high coverage but body text blocks are overly fragmented.

When standalone Markdown, custom dump review, or reuse of an existing layout dump is needed, use the following four-step method (the main workflow already performs the equivalent work automatically by default):

# 1. 获取文字真值（--dump-and-pdf 同时出 dump 和双层 PDF）
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf \
  --backend paddle_api --paddle-model PP-OCRv6 \
  --ocr-dump /tmp/text.json --dump-and-pdf

# 2. 获取版面结构
python3 scripts/pdf-ocr.py -i input.pdf -o unused.pdf \
  --backend paddle_api --paddle-model PP-StructureV3 --ocr-dump /tmp/layout.json

# 3. 生成自然段文本，同时输出不含正文的诊断和已过滤文字层 dump
python3 scripts/pdf_ocr_paragraphs.py \
  --text-dump /tmp/text.json --layout-dump /tmp/layout.json \
  --output /tmp/clean.md --diagnostics /tmp/paragraphs.json \
  --filtered-dump /tmp/text-filtered.json

# 4. 用过滤后的行级坐标生成双层 PDF
python3 scripts/pdf-ocr.py -i input.pdf -o output.pdf \
  --backend paddle_api --ocr-resume /tmp/text-filtered.json

In diagnostics, when layout_coverage is below 0.75 , it automatically falls back to pure geometric rules; at the same time check structure_text_fallbacks and layout_boundary_mergesto confirm that both low-confidence replacement and cross-block merging are auditable. Only when Structure clearly misses blocks, has incorrect region types, or has incorrect reading order, and the pure geometric fallback still cannot recover, test PaddleOCR-VL-1.5 as --layout-dump; VL-1.6 is not used as the default fallback. Sensitive information such as full text, dumps, and account credentials continues to be placed only in temporary directories unless the user explicitly requests archiving.

Individual tools

# 手动旋转
python3 scripts/pdf-rotate.py --input input.pdf --output output.pdf --angle 90

# 解密
python3 scripts/pdf-decrypt.py --input input.pdf --output output.pdf
python3 scripts/pdf-decrypt.py --input input.pdf --output output.pdf --password 123456

# 去水印
python3 scripts/pdf-remove-watermark.py --input input.pdf --output output.pdf

# 压缩
python3 scripts/pdf-compress.py -i input.pdf -o output.pdf --level medium

# 加页码
python3 scripts/pdf-add-page-numbers.py -i input.pdf -o output.pdf

# 合并
python3 scripts/pdf-merge.py -i file1.pdf file2.pdf file3.pdf -o merged.pdf
python3 scripts/pdf-merge.py -i file1.pdf file2.pdf -o merged.pdf --add-numbers --continuous

For detailed parameters such as page numbers, merging, and compression, see references/pdf-workflows.md。

Dependencies

Base dependencies

pip install pymupdf pypdf pillow numpy opencv-python pdf2image

macOS:

brew install poppler

Linux:

sudo apt-get install poppler-utils

OCR fallback dependencies

pip install ocrmypdf

macOS:

brew install tesseract tesseract-lang

Linux:

sudo apt-get install tesseract-ocr tesseract-ocr-chi-sim

Local RapidOCR (recommended, local-first engine)

pip install rapidocr

After installation, auto 's local fallback and --local-only both prefer RapidOCR (local ONNX inference; Chinese line-level recognition quality is significantly higher than tesseract, and nothing leaves the local machine); on first run, detection/recognition models are downloaded automatically (about 5-16MB each, one time only). You can also explicitly specify --backend rapidocr_local。

For the complete list of optional dependencies, see references/optional-dependencies.txt. The historically retained local Paddle dual-layer implementation has been split out to scripts/pdf_ocr_paddle_local.pyand is not part of the default production pipeline; install paddleocr paddlepaddle only when experimentation is needed and integrate it separately.

Quality checks

python3 scripts/pdf-ocr-quality-check.py \
  -i input.pdf -o output.pdf --keywords 合同,法院

python3 scripts/pdf-ocr-benchmark.py \
  -i input.pdf \
  --backend local_ocrmypdf \
  --sample-pages 5 \
  --skip-coarse-rotation \
  --preprocess-jobs 6 \
  --preprocess-chunk-pages 80

The keyword gate first performs NFKC, case, and whitespace normalization to avoid Chinese OCR being misjudged as a miss after spaces are inserted between Chinese characters; CER is still computed against independent reference text.

For common questions, see references/troubleshooting.md。

Pre-delivery checks

1. Confirm that the number of output pages matches the original file.
2. Spot-check page orientation, clarity, cropping boundaries, and file size.
3. For dual-layer PDFs, test text search, copying, and keyword hits.
4. Inform the user of the backend actually used, the output file paths, and any fallback situations.

PaddleOCR coordinates and rotated pages (v2.11.1+)

PaddleOCR API line-level poly uses pixel coordinates with the top-left origin of the forward-rendered image; PyMuPDF page coordinates likewise use the top-left origin. Do not guess the y origin from text distribution, and do not perform page_height - y flipping.

If the input PDF relies on rotation metadata to be oriented correctly, the Paddle path first uses PyMuPDF to losslessly remove rotation, transforming the original page content into rotation=0 an upright page, and then overlays according to the original scaling and text layout logic. This process does not resample the scanned image and does not change the visible page pixels. Backends such as MinerU whose coordinate contract has not yet been verified continue to maintain their original behavior and do not reuse Paddle's coordinate assumptions.