SuperAPI AI Directory

wigolo - An open-source local search tool that supports integration with multiple coding agents

Author: AI小集 · Original source: AI工具集

What is wigolo

wigolo is an open-source local search tool that integrates via MCP with Claude Code、Cursor and other coding agents. With no API Key required, it can perform search, web scraping, full-site crawling, structured extraction, caching, and research locally at zero cost. The tool supports parallel queries across 18 search engines, with results accompanied by explainable scores and source citations, and all data is stored locally.

wigolo

wigolo's main features

  • search: Calls 18 search engines in parallel, returning search results with explainable scores via on-device ML reranking.
  • fetch: Fetches web pages via tiered routing, automatically escalating from pure HTTP to a headless browser, intelligently handling anti-scraping measures and SPA pages.
  • crawl: Supports full-site crawling in BFS, DFS, and sitemap modes, respects robots.txt, and automatically deduplicates.
  • extract: Extracts tables, metadata, JSON-LD, and structured data with custom JSON schemas from pages.
  • cache: Stores fetched content in a local index, supports keyword and semantic retrieval, and provides millisecond-level responses for repeated queries with offline availability.
  • find_similar: Through a three-way fusion of keywords, semantics, and real-time web pages, finds pages similar to a target URL or concept.
  • research: Automatically decomposes complex questions into subqueries, fetches resources in parallel, and synthesizes a structured research report with cited sources.
  • agent: Executes an autonomous collection loop of "plan -> search -> fetch -> extract -> synthesize", with step logs and a time budget.
  • diff + watch: Monitors changes to pages since the last visit and pushes change results to a specified webhook.
Dig Dig GitHub

Follow on WeChat and reply "open source" to join theAI Open Source Project Exchange Group

How to use wigolo

  • Environment preparation: Make sure Node.js 20 or later is installed locally.
  • One-click installation: Run npx wigolo init to automatically download the browser engine and models, and integrate with Claude Code、Cursor and other agents.
  • Health check: Execute npx wigolo health to verify whether search, the browser, and on-device models are running properly.
  • Optional LLM configuration: If you need research or agent features, set a free Gemini API Key or a local Ollama model.
  • Invoke in an Agent: In Claude Code, Cursor, and other MCP-supported agents, directly invoke tools such as search and scraping through natural language.
  • Direct CLI use: Run commands such as npx wigolo search/fetch/crawl from the command line to perform specific operations.
  • Local cache reuse: All data is automatically stored in the ~/.wigolo/ directory, supporting offline reuse and semantic retrieval.

wigolo's core advantages

  • Local-first: All search, scraping, and caching are completed on the local machine, data never leaves the device, and it works normally even when offline.
  • Zero cost and zero barrier: No account registration and no API Key required, every query is permanently free, and it can be integrated with mainstream agents with a single command.
  • Fully explainable results: Each search result includes byte-level source excerpts, citation IDs, and score breakdowns, making quality clear at a glance.
  • Multi-engine parallel search: Queries 18 search engines simultaneously, using rank fusion and on-device ML reranking to ensure comprehensive and precise results.
  • Intelligent Tiered Crawling: Automatically upgrades from plain HTTP to TLS impersonation and headless browsers, honestly handling anti-crawling, and clearly reporting failures.
  • Persistent Local Cache: All content is stored in a local hybrid index (keywords + semantic vectors), with millisecond-level responses for repeated queries.

wigolo's project address

  • Project Website:https://knockoutez.github.io/wigolo/
  • GitHub Repository:https://github.com/KnockOutEZ/wigolo

Comparison of wigolo with similar competitors

Comparison Dimension Wigolo Firecrawl
Project Positioning Local-first AI Agent web intelligence layer Cloud-based web scraping and structured extraction platform
Core Proposition Search, scraping, and research all run locally, at zero cost Convert any website into structured data usable by LLMs
Developer @KnockOutEZ (personal open source) Mendable (venture-backed startup)
License AGPL-3.0 (permanently open source and free) Closed-source commercial product
Search Capability ✅ 18 search engines in parallel, on-device ML reranking, results with explainable scores ❌ No built-in search, requires pairing with other search APIs
Web Scraping ✅ Tiered routing (HTTP → TLS impersonation → headless browser) ✅ Powerful scraping capability, supports dynamic rendering
Full-Site Crawling ✅ BFS/DFS/sitemap, complies with robots.txt ✅ Supports full-site crawling and sitemap generation
Structured Extraction ✅ Tables, metadata, JSON-LD, custom JSON schemas ✅ Supports LLM extraction and structured data output
Local Cache ✅ Persistent local index (keywords + semantic vectors), available offline, repeated queries free ❌ No local cache, every scrape goes through cloud API
Result Explainability ✅ Byte-level original text excerpts + citation IDs + score breakdown ❌ Returns structured data, with no per-item scoring and source locating

wigolo's application scenarios

  • AI Coding Assistance: The Agent automatically searches the latest API documentation, official tutorials, and best practices while writing code, with no need to manually switch browsers.
  • Technical Research and Competitive Analysis: Let the Agent autonomously search and scrape multiple technical blogs and documentation sites, and synthesize a structured research report with citations.
  • Full-Site Data Scraping: Batch-crawl product catalogs, pricing information, paper libraries, or news sites, and extract tables and structured data locally.
  • Local Knowledge Base Construction: Cache frequently consulted documents and web pages locally to form a personal/team offline searchable knowledge base.
  • Page Change Monitoring: Continuously monitor competitor official websites, documentation update pages, or pricing pages, and automatically notify via webhook when changes occur.