What is wigolo
wigolo is an open-source local search tool that integrates via MCP with Claude Code、Cursor and other coding agents. With no API Key required, it can perform search, web scraping, full-site crawling, structured extraction, caching, and research locally at zero cost. The tool supports parallel queries across 18 search engines, with results accompanied by explainable scores and source citations, and all data is stored locally.
wigolo's main features
-
search: Calls 18 search engines in parallel, returning search results with explainable scores via on-device ML reranking.
-
fetch: Fetches web pages via tiered routing, automatically escalating from pure HTTP to a headless browser, intelligently handling anti-scraping measures and SPA pages.
-
crawl: Supports full-site crawling in BFS, DFS, and sitemap modes, respects robots.txt, and automatically deduplicates.
-
extract: Extracts tables, metadata, JSON-LD, and structured data with custom JSON schemas from pages.
-
cache: Stores fetched content in a local index, supports keyword and semantic retrieval, and provides millisecond-level responses for repeated queries with offline availability.
-
find_similar: Through a three-way fusion of keywords, semantics, and real-time web pages, finds pages similar to a target URL or concept.
-
research: Automatically decomposes complex questions into subqueries, fetches resources in parallel, and synthesizes a structured research report with cited sources.
-
agent: Executes an autonomous collection loop of "plan -> search -> fetch -> extract -> synthesize", with step logs and a time budget.
-
diff + watch: Monitors changes to pages since the last visit and pushes change results to a specified webhook.
Follow on WeChat and reply "open source" to join theAI Open Source Project Exchange Group
How to use wigolo
-
Environment preparation: Make sure Node.js 20 or later is installed locally.
-
One-click installation: Run
npx wigolo initto automatically download the browser engine and models, and integrate with Claude Code、Cursor and other agents. -
Health check: Execute
npx wigolo healthto verify whether search, the browser, and on-device models are running properly. -
Optional LLM configuration: If you need research or agent features, set a free Gemini API Key or a local Ollama model.
-
Invoke in an Agent: In Claude Code, Cursor, and other MCP-supported agents, directly invoke tools such as search and scraping through natural language.
-
Direct CLI use: Run commands such as
npx wigolo search/fetch/crawlfrom the command line to perform specific operations. -
Local cache reuse: All data is automatically stored in the
~/.wigolo/directory, supporting offline reuse and semantic retrieval.
wigolo's core advantages
-
Local-first: All search, scraping, and caching are completed on the local machine, data never leaves the device, and it works normally even when offline.
-
Zero cost and zero barrier: No account registration and no API Key required, every query is permanently free, and it can be integrated with mainstream agents with a single command.
-
Fully explainable results: Each search result includes byte-level source excerpts, citation IDs, and score breakdowns, making quality clear at a glance.
-
Multi-engine parallel search: Queries 18 search engines simultaneously, using rank fusion and on-device ML reranking to ensure comprehensive and precise results.
-
Intelligent Tiered Crawling: Automatically upgrades from plain HTTP to TLS impersonation and headless browsers, honestly handling anti-crawling, and clearly reporting failures.
-
Persistent Local Cache: All content is stored in a local hybrid index (keywords + semantic vectors), with millisecond-level responses for repeated queries.
wigolo's project address
- Project Website:https://knockoutez.github.io/wigolo/
- GitHub Repository:https://github.com/KnockOutEZ/wigolo
Comparison of wigolo with similar competitors
| Comparison Dimension | Wigolo | Firecrawl |
|---|---|---|
| Project Positioning | Local-first AI Agent web intelligence layer | Cloud-based web scraping and structured extraction platform |
| Core Proposition | Search, scraping, and research all run locally, at zero cost | Convert any website into structured data usable by LLMs |
| Developer | @KnockOutEZ (personal open source) | Mendable (venture-backed startup) |
| License | AGPL-3.0 (permanently open source and free) | Closed-source commercial product |
| Search Capability | ✅ 18 search engines in parallel, on-device ML reranking, results with explainable scores | ❌ No built-in search, requires pairing with other search APIs |
| Web Scraping | ✅ Tiered routing (HTTP → TLS impersonation → headless browser) | ✅ Powerful scraping capability, supports dynamic rendering |
| Full-Site Crawling | ✅ BFS/DFS/sitemap, complies with robots.txt | ✅ Supports full-site crawling and sitemap generation |
| Structured Extraction | ✅ Tables, metadata, JSON-LD, custom JSON schemas | ✅ Supports LLM extraction and structured data output |
| Local Cache | ✅ Persistent local index (keywords + semantic vectors), available offline, repeated queries free | ❌ No local cache, every scrape goes through cloud API |
| Result Explainability | ✅ Byte-level original text excerpts + citation IDs + score breakdown | ❌ Returns structured data, with no per-item scoring and source locating |
wigolo's application scenarios
-
AI Coding Assistance: The Agent automatically searches the latest API documentation, official tutorials, and best practices while writing code, with no need to manually switch browsers.
-
Technical Research and Competitive Analysis: Let the Agent autonomously search and scrape multiple technical blogs and documentation sites, and synthesize a structured research report with citations.
-
Full-Site Data Scraping: Batch-crawl product catalogs, pricing information, paper libraries, or news sites, and extract tables and structured data locally.
-
Local Knowledge Base Construction: Cache frequently consulted documents and web pages locally to form a personal/team offline searchable knowledge base.
-
Page Change Monitoring: Continuously monitor competitor official websites, documentation update pages, or pricing pages, and automatically notify via webhook when changes occur.