Feature Description
This skill provides comprehensive web search and web content retrieval capabilities.
Available Tools
1. WebFetch (OpenClaw built-in)
- Function: Retrieve web content
- Use case: Retrieve the content of a specific webpage
2. Tavily Skill (AI-optimized search)
- Location:
~/clawd/skills/tavily/ - Function: AI-optimized web search
- Command:
~/clawd/skills/tavily/scripts/tavily.sh
Tavily usage
# 基础搜索
~/clawd/skills/tavily/scripts/tavily.sh search <query> [max_results]
# 深度搜索
~/clawd/skills/tavily/scripts/tavily.sh search <query> --deep
# 搜索并提取
~/clawd/skills/tavily/scripts/tavily.sh extract <query> [max_results]
3. Firecrawl Skill (Professional scraping)
- Location:
~/clawd/skills/firecrawl/ - Function: Professional web scraping and data extraction
- Command:
~/clawd/skills/firecrawl/scripts/firecrawl.sh
Firecrawl usage
# 抓取单个网页
~/clawd/skills/firecrawl/scripts/firecrawl.sh scrape <url> [format]
# 搜索并抓取
~/clawd/skills/firecrawl/scripts/firecrawl.sh search <query> [limit]
# 批量爬取
~/clawd/skills/firecrawl/scripts/firecrawl.sh crawl <url> [max_pages]
Use Cases
When to use Tavily
- Search the internet for information
- Get real-time data
- Need AI-optimized search results
When to use WebFetch
- Simply retrieve web content
- Quickly view page information
When to use Firecrawl
- Complex web scraping
- Structured data extraction
- Batch website crawling
Examples
Search for Polymarket information (Tavily)
~/clawd/skills/tavily/scripts/tavily.sh search "Polymarket Fed decision March 2026" 5
Scrape a specific page (Firecrawl)
~/clawd/skills/firecrawl/scripts/firecrawl.sh scrape "https://polymarket.com/event/fed-decision-in-march-885"
Feature Description
This skill provides comprehensive web search and web content retrieval capabilities, including:
- Internet search (news, information, data)
- Web content scraping and parsing
- Real-time data queries
- Multi-source search result integration
Available Tools
1. OpenClaw built-in WebSearch
- Function: Internet search
- Use case: Search news, information, real-time data
- Trigger: When the user asks a question requiring real-time information
2. OpenClaw built-in WebFetch
- Function: Retrieve web content
- Use case: Retrieve the content of a specific webpage
- Trigger: When the user provides a URL and requests its content
3. Firecrawl MCP
- Function: Professional web scraping and data extraction
- Tool: mcp__firecrawl__scrape, mcp__firecrawl__crawl, mcp__firecrawl__search
- Use case: Complex web scraping, structured data extraction, batch crawling
- Advantages: Supports complex webpages, dynamic content, structured output
Use Cases
When to use WebSearch
- "Search for the latest AI news"
- "Find the latest Polymarket odds"
- "What's the weather like today"
- "A company's latest stock price"
When to use WebFetch
- "Retrieve the content of this webpage"
- "Read the content of this URL"
- "What does this page say"
When to use Firecrawl MCP
- "Scrape all product information from this website"
- "Extract structured data from this webpage"
- "Crawl multiple pages of this website"
Workflow
- Analyze requirements: Determine whether to search or scrape
- Choose tools:
- Simple search → WebSearch
- Single-page scraping → WebFetch or mcp__firecrawl__scrape
- Complex scraping → mcp__firecrawl__crawl
- Perform the operation: Call the corresponding tool
- Integrate results: Integrate the search results into the answer
Example
Search for information
用户: "搜索一下最新的 AI 新闻"
操作: 使用 WebSearch 搜索 "AI news latest"
Retrieve webpage content
用户: "获取 https://example.com 的内容"
操作: 使用 WebFetch 或 mcp__firecrawl__scrape
Structured data extraction
用户: "从这个网页提取所有产品价格"
操作: 使用 mcp__firecrawl__scrape 并指定提取格式
Notes
- Search results may contain outdated or inaccurate information and need to be verified
- Web scraping must comply with the website's robots.txt and terms of use
- Paid content or pages that require login may not be directly accessible
- Large-scale scraping may trigger anti-scraping mechanisms