glmocr-formula

GLM-OCR Formula Recognition Skill / GLM-OCR Formula Recognition Skill

Recognize mathematical formulas from images and PDFs and convert them to LaTeX format using the ZhiPu GLM-OCR layout parsing API.

When to Use / When to Use

Extract mathematical formulas from images or scanned documents / Extract mathematical formulas from images or scanned documents
Convert formula images to LaTeX / Convert formula images to LaTeX
Recognize complex equations, integrals, matrices / Recognize complex equations, integrals, matrices
Parse scientific papers, textbooks, exam papers with formulas / Parse scientific papers, textbooks, exam papers with formulas
User mentions "formula OCR", "extract formula", "formula recognition", "formula OCR", "extract formula", "image to LaTeX"

Key Features / Key Features

Complex formula support: Handles integrals, summations, matrices, fractions, radicals
LaTeX output: Formulas are output in LaTeX format, ready for use in documents
Inline & block formulas: Recognizes both inline and display-style formulas
Mixed content: Can handle documents with both text and formulas
Local file & URL: Supports both local files and remote URLs

Resource Links / Resource Links

Resource	Link
Get API Key	Zhipu Open Platform API Keys
API Docs	Layout Parsing / Layout Parsing

Prerequisites / Prerequisites

API Key Setup / API Key Setup (Required / Required)

The script retrieves the key via the ZHIPU_API_KEY environment variable, and can reuse the same key with other Zhipu skills. This script reads the key from the ZHIPU_API_KEY environment variable. Reusing the same key across Zhipu skills is optional.

Get Key / Get Key: Visit Zhipu Open Platform API Keys to create or copy your key.

Setup options / Setup options (choose one):

Global config (recommended) / Global config (recommended): Set once in openclaw.json under env.vars, all Zhipu skills will share it:
```
{
  "env": {
    "vars": {
      "ZHIPU_API_KEY": "你的密钥"
    }
  }
}
```

Skill-level config / Skill-level config: Set for this skill only in openclaw.json:

{
  "skills": {
    "entries": {
      "glmocr-formula": {
        "env": {
          "ZHIPU_API_KEY": "你的密钥"
        }
      }
    }
  }
}

Shell environment variable / Shell environment variable: Add to ~/.zshrc:
```
export ZHIPU_API_KEY="你的密钥"
```

💡 If you have already configured a key for other Zhipu skills (such as glmocr、glmv-caption、glm-image-generation), they share the same ZHIPU_API_KEY, no need to configure again.

Security & Transparency / Security & Transparency

Environment variables used / Environment variables used:
- ZHIPU_API_KEY (required / required)
- GLM_OCR_TIMEOUT (optional timeout seconds / optional timeout seconds)
Fixed endpoint / Fixed official endpoint: https://open.bigmodel.cn/api/paas/v4/layout_parsing
No custom API URL override / No custom API URL override: avoids accidental key exfiltration via redirected endpoints.
Raw upstream response is optional / Raw response is not returned by default: use --include-raw only when needed for debugging.

⛔ MANDATORY RESTRICTIONS / MANDATORY RESTRICTIONS ⛔

ONLY use GLM-OCR API — Execute the script python scripts/glm_ocr_cli.py
NEVER parse formulas yourself — Do NOT try to extract formulas using built-in vision or any other method
NEVER offer alternatives — Do NOT suggest "I can try to read it" or similar
IF API fails — Display the error message and STOP immediately
NO fallback methods — Do NOT attempt formula extraction any other way

📋 Output Display Rules / Output Display Rules

After running the script, present the OCR result clearly and safely.

Show extracted text/formulas (text) in full
Summarization is allowed, but do not hide important extraction failures
If layout_details contains formula-related entries, you may highlight them
If the result file is saved, tell the user the file path
Show raw upstream response only when explicitly requested or debugging (--include-raw)

⚠️ LaTeX Rendering / LaTeX Rendering Note:

OCR API returns formulas in LaTeX format (e.g., $\frac{1}{2}$ , $\theta^{x+1}$ ). Since most chat platforms do not render LaTeX, you should ask the user once (on first use):

"The OCR result contains LaTeX formulas. Do you want me to convert the formulas to Unicode readable format for display, or keep the original LaTeX?"

Remember the user's choice for the rest of the session. Do NOT ask again on subsequent calls unless the user explicitly changes their preference.

User chooses readable format → convert LaTeX to Unicode/plain-text:

LaTeX	Unicode / plain text
$\frac{a}{b}$	a/b
$x^{n}$	x^n
$x_{i}$	xᵢ
$\sqrt{x}$	√x
$\theta$	θ
$\phi$	φ
$\therefore$	∴
$\Rightarrow$	⇒
$\left\{ \begin{array}{l} ... \end{array} \right.$	⎧ line1 ⎨ line2 ⎩
$\textcircled{1}$	①
$\in$	∈
$\infty$	∞
$\ln$	ln
$\leq$ / $\geq$	≤ / ≥

User chooses raw LaTeX → display the original LaTeX output directly, and remind them the raw data is also saved in the output file if --output was used.

How to Use / How to Use

Extract from URL / Extract from URL

python scripts/glm_ocr_cli.py --file-url "https://example.com/formula.png"

Extract from Local File / Extract from Local File

python scripts/glm_ocr_cli.py --file /path/to/equation.png

Save Result to File / Save Result to File

python scripts/glm_ocr_cli.py --file formula.png --output result.json --pretty

Include Raw Upstream Response (Debug Only) / Include Raw Upstream Response (Debug Only)

python scripts/glm_ocr_cli.py --file formula.png --output result.json --include-raw

CLI Reference / CLI Parameters

python {baseDir}/scripts/glm_ocr_cli.py (--file-url URL | --file PATH) [--output FILE] [--pretty] [--include-raw]

Parameter	Required	Description
`--file-url`	One of	URL to image/PDF
`--file`	One of	Local file path to image/PDF
`--output`, `-o`	No	Save result JSON to file
`--pretty`	No	Pretty-print JSON output
`--include-raw`	No	Include raw upstream API response in `result` field (debug only)

Response Format / Response Format

{
  "ok": true,
  "text": "Extracted formulas and text in Markdown/LaTeX...",
  "layout_details": [...],
  "result": null,
  "error": null,
  "source": "/path/to/file",
  "source_type": "file",
  "raw_result_included": false
}

Key fields:

ok — whether extraction succeeded
text — extracted text in Markdown with LaTeX formulas
layout_details — layout analysis details
error — error details on failure

Error Handling / Error Handling

API key not configured:

ZHIPU_API_KEY not configured. Get your API key at: https://bigmodel.cn/usercenter/proj-mgmt/apikeys

→ Show exact error to user, guide them to configure

Authentication failed (401/403): API key invalid/expired → reconfigure

Rate limit (429): Quota exhausted → inform user to wait

File not found: Local file missing → check path