article-extractor

Article Extractor

Extract clean article content from URLs, removing ads, navigation, and clutter. Multi-tool fallback ensures reliability.

Workflow

When user provides a URL to download/extract:

1. Call the extraction script directly with the URL (do NOT fetch the URL first with web_fetch)
2. Script handles fetching, extraction, and saving automatically
3. Returns clean markdown file with frontmatter

Usage

# Basic extraction
scripts/extract-article.sh "https://example.com/article"

# Specify output location
scripts/extract-article.sh "https://example.com/article" -o my-article.md -d ~/Documents

Try Wayback Machine if original fails

scripts/extract-article.sh "https://example.com/article" --wayback

Make script executable if needed: `chmod +x scripts/extract-article.sh`

## Key Options

- `-o <file>` - Output filename
- `-d <dir>` - Output directory
- `-w, --wayback` - Try Wayback Machine if extraction fails
- `-t <tool>` - Force tool: `jina`, `trafilatura`, `readability`, `fallback`
- `-q` - Quiet mode

For complete options, exit codes, tool details, and examples, see [references/tools-and-options.md](references/tools-and-options.md).

## Common Failures

- **Exit 3 (access denied)**: Paywall or login required - try `--wayback`
- **Exit 4 (no content)**: Heavy JavaScript - try different `--tool`
- **Exit 2 (network)**: Connection issue - check URL

## Local Tools (Optional)

For offline extraction: `scripts/install-deps.sh`