deep-research

Deep Research

Execute multi-step autonomous research: market analysis, competitor research, industry reports, literature reviews. Automatically search, analyze, and synthesize to generate a complete research report.

Use Cases

The user says "Help me research the AI office assistant market" "Analyze the top 5 competitors"
The user says "Make an industry trends report" "Research the opportunities in this sector"
The user says "Help me conduct an in-depth study of this topic and write a complete report"
The user says "Collect and organize the hot news and information in the AI industry over the past week"

Execution Method

Use crawler-type Skills (such as Crawl4AI) to quickly obtain complete webpage content, greatly shortening research time.

Research Process

Step 1: 理解调研目标
  ↓ 明确范围、深度、输出格式

Step 2: 制定调研计划
  ↓ 拆解为 3-5 个子课题

Step 3: 批量搜索 + 内容抓取 (核心)
  ↓ 3.1 调用 web_search 工具获取相关 URL 列表 (自动选择 Tavily/Exa/Jina)
  ↓ 3.2 爬虫类 Skill (Crawl4AI) 并发抓取完整内容
  ↓     Playwright 浏览器引擎 → 突破反爬
  ↓     PruningContentFilter → 去除噪声
  ↓     自动输出干净 Markdown

Step 4: 交叉验证
  ↓ 多个来源互相印证

Step 5: 综合分析
  ↓ 发现趋势、对比、洞察

Step 6: 生成报告
  ↓ 结构化输出

Implementation Example

from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, CacheMode
from crawl4ai.content_filter_strategy import PruningContentFilter
from crawl4ai.markdown_generation_strategy import DefaultMarkdownGenerator

# Step 1: 调用 web_search 工具获取 URL（自动选择最佳搜索源）
search_queries = ["AI 办公助手 市场分析", "AI 办公助手 竞品对比"]

all_urls = []
for query in search_queries:
    # 直接调用 web_search 工具（自动降级 Tavily → Exa → Jina）
    results = await web_search(query=query, max_results=10, search_depth="advanced")
    all_urls.extend([r["url"] for r in results.get("results", [])[:5]])

unique_urls = list(set(all_urls))[:15]

# Step 2: Crawl4AI 并发抓取完整内容
config = CrawlerRunConfig(
    cache_mode=CacheMode.BYPASS,
    markdown_generator=DefaultMarkdownGenerator(
        content_filter=PruningContentFilter(threshold=0.4)
    ),
)

async with AsyncWebCrawler() as crawler:
    results = await crawler.arun_many(unique_urls, config=config)

# Step 3: 提取成功的文章
valid = [r for r in results if r.success]

# Step 4: 构建上下文给 LLM 分析
context = ""
for article in valid:
    content = article.markdown.fit_markdown or article.markdown.raw_markdown
    context += f"来源: {article.url}\n\n"
    context += f"{content[:2000]}\n\n---\n\n"

# Step 5: LLM 综合分析 (基于完整内容，质量远高于搜索摘要)

Report Structure

# [调研主题] 调研报告

**调研日期**: YYYY-MM-DD
**调研范围**: [描述]

## Executive Summary
[1-2 段核心发现]

## 1. 背景与现状
[行业/市场背景]

## 2. 主要发现
### 2.1 [子课题 1]
[详细分析]

### 2.2 [子课题 2]
[详细分析]

## 3. 对比分析
[表格对比、优劣势分析]

## 4. 趋势与预测
[基于数据的趋势判断]

## 5. 建议与行动项
[可执行的建议]

## 参考来源
[标注所有信息来源 URL]

Research Quality Standards

Dimension	Requirement
Source Diversity	At least 5 different sources
Timeliness	Prioritize data from the most recent 12 months
Cross-Validation	Key data confirmed by at least 2 sources
Objectivity	Present multiple perspectives, without bias
Traceability	All data labeled with sources
Content Completeness	Based on complete articles (not search snippets)

Output Specifications

Report length adapts to topic complexity (1000-5000 words)
Use tables and comparison matrices to enhance readability
All data labeled with source links
Clearly distinguish between "facts" and "analysis/speculation"
Attach a list of information sources at the end of the report