document-privacy-audit

Document Privacy Audit

Overview

Perform full-text extraction and structured retrieval, combine rule-based and semantic judgment to locate private information, and output a summary report with locations and risk levels.

Workflow

1. Define scope and deliverables

• Confirm the file path and format, language, and whether scanned pages are included.
• Clarify the privacy scope (PII only, or also including financial/health/credentials).
• Agree on the output format (Markdown report + JSON details, or report only).

1. Extract the full text and preserve locations

• Prefer using scripts/extract_text.py to output JSONL, preserving page/paragraph/cell locations.
• If a PDF has no text layer, prompt to use OCR (ocrmypdf or tesseract) before extraction.

Example:

python scripts/extract_text.py --input /path/to/file.pdf --output /path/to/extracted.jsonl
python scripts/extract_text.py --input /path/to/file.docx --output /path/to/extracted.jsonl

1. Full-text retrieval and structured understanding

• Perform keyword retrieval on the extraction results (rg or a custom script), preserving hit locations.
• Summarize the document structure and topics to provide context for privacy judgment (for example, whether a table is a roster/bill/medical record).

1. Identify private information

• Use scripts/pii_scan.py to perform rule scanning and generate a candidate list.
• Conduct semantic review in context, remove obvious false positives, and supplement missed detections.
• For classification and grading, please refer to references/pii_catalog.md。

Example:

python scripts/pii_scan.py --input /path/to/extracted.jsonl --output /path/to/pii_report.json

1. Summary output

• Output an audit summary, detailed findings, risk levels, and recommendations.
• All examples must be displayed in masked form, preserving location information for review.

Output template

# 隐私审计报告
- 文件: <path>
- 扫描范围: <PII/财务/健康/凭据>
- 扫描时间: <YYYY-MM-DD>
- 结论: <低/中/高风险 + 1-2 句说明>

## 发现摘要
| 类别 | 数量 | 示例(脱敏) | 位置 |
| --- | --- | --- | --- |
| 邮箱 | 3 | a***@example.com | page:2, page:5 |
| 手机 | 2 | 13******45 | page:3 |

## 高风险项
- <类别 + 脱敏示例 + 位置 + 风险原因>

## 细项清单
- <类别> | <脱敏示例> | <位置> | <上下文简述>

## 建议
- <脱敏/删除/分级访问/加密存储等>

Resources

scripts/

• extract_text.py: Uniformly extract PDF/Office text and preserve locations.
• pii_scan.py: Rule-scan private information and generate a JSON report.

references/

• pii_catalog.md: Privacy categories, risk levels, and masking examples.