SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillscrawl-xueqiu-my-timeline

crawl-xueqiu-my-timeline

Writing & research

View repository

Updated 10-15 19:50

About this skill

crawl-xueqiu-my-timeline

Crawl the timeline of followed users on the Xueqiu homepage, save it as Markdown files grouped by speaker, for AI to analyze and summarize and generate a PDF investment report.

Features：

  • Automatically filter out official accounts (market updates from listed companies, indices, ETFs, etc.)
  • Output grouped by speaker, sorted in descending order by number of posts
  • Within each group, sorted in reverse chronological order, with newer content first

Quick Start

# 1. 检查环境
sh skills/crawl-xueqiu-my-timeline/scripts/check-cdp.sh
sh skills/crawl-xueqiu-my-timeline/scripts/check-agent-browser.sh

# 2. 爬取最近 24 小时时间线
skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py
3. AI automatically reads, analyzes, and generates a PDF report 

## 何时使用

✅ **应该触发**：
- "帮我看看雪球上今天有什么投资要闻"
- "生成一份市场热点分析报告"
- "把关注的雪球大 V 观点整理成 PDF"
- "最近有什么投资热点"
- "爬取雪球时间线"
- "生成投资分析报告"

❌ **不应触发**：
- "打开雪球网站"（只需浏览器操作）
- "查看某只股票行情"（查询单一数据）
- "帮我登录雪球"（简单网页操作）

## 前置准备

确保 Chrome 处于 Debug 模式并安装 agent-browser：

```bash
sh skills/crawl-xueqiu-my-timeline/scripts/check-cdp.sh
sh skills/crawl-xueqiu-my-timeline/scripts/check-agent-browser.sh

AI Task List

  1. ✅ Check environment: Chrome Debug mode + agent-browser
  2. ✅ Run crawl: Run crawl_xueqiu_home_timeline_api.py
  3. ✅ Read files: Read the generated home_timeline_YYYYMMDD_YYYYMMDD.md
  4. ✅ Create TODO: Create an analysis TODO list based on the number of speakers
  5. ✅ Distribute tasks: Launch subagents to analyze user posts (concurrency ≤ 3)
  6. ✅ Aggregate results: Integrate the analysis results of all subagents
  7. ✅ Generate report: Write a Markdown-format investment analysis report
  8. ✅ Convert to PDF: Use mdpdf to convert to PDF
  9. ✅ Clean up files: Delete temporary Markdown files

AI Analysis Guidelines

⚠️ Important: Determining attribution of opinions

Only content that does not start with // or > is the speaker's own opinion：

Marker Meaning Summarize?
Unmarked line The speaker's own post ✅ Needs summary
// starts with Another user's post being commented on ❌ Not the person's own opinion
> starts with Quoted reposted content ❌ Not the person's own opinion

Note：回复@XXX： The prefix is automatically generated by Xueqiu; the entire line is the speaker's own opinion and should be summarized.

Example：

回复@阿企笔记：要知道很多农村的人需要方言语音设备
//回复@青年完：支付宝本来就有音响
> @阿企笔记：中概股失去增长动力

Correct summary: The speaker believes rural users need dialect voice devices

Incorrect summary: The speaker discussed Alipay speakers and Chinese concept stock earnings reports

Read output files

The crawler script will generate Markdown files in the current directory:home_timeline_YYYYMMDD_YYYYMMDD.md

# 读取最新生成的时间线文件
cat home_timeline_*.md

Analysis Report Format

AI should generate the investment analysis report according to the following structure:

# 雪球关注时间线分析报告

## 时间范围
2026-03-05 至 2026-03-06

## 发言人统计
- @产业驱动博弈：19 条
- @管我财：6 条
...（逐一列出所有发言人）

## 发言观点总结

### @发言人 A（发言最多的用户）
- **发言数量**: X 条
- **主要观点**:
  - 观点 1 摘要
  - 观点 2 摘要
- **涉及标的**: $股票 1$, $股票 2$
- **情绪倾向**: 乐观/中性/悲观

### @发言人 B
- **发言数量**: X 条
- **主要观点**:
  - 观点摘要
- **涉及标的**: $股票 3$
- **情绪倾向**: 乐观/中性/悲观

...（继续列出所有发言人，直到最后）...

## 市场热点 TOP3
1. 热点主题 1 - 讨论人数 X
2. 热点主题 2 - 讨论人数 X
3. 热点主题 3 - 讨论人数 X

## 投资建议/风险提示
（基于发言内容的综合分析）

Core Principles：

  • ✅ Must include all speakers, even if there is only 1 post
  • ✅ Sort in descending order by number of posts
  • ✅ Only summarize the person's own opinions (content that does not start with // or > )
  • ✅ Mark speakers whose analysis failed as "⚠️ Analysis failed" in the report

Analysis Dimensions

  1. Group by speaker: Aggregate all posts by the same user
  2. Opinion extraction: Identify core opinions, deduplicate and merge similar content
  3. Ticker identification: Extract $股票名$ format code
  4. Sentiment analysis: Determine market sentiment based on word usage
  5. Hot topic aggregation: Count topics discussed across users

AI analysis workflow (Subagent scheduling)

Step 1: Create a TODO list

After reading the timeline file, count the number of messages from each speaker, and based onthe number of messages from a single userdecide the TODO allocation strategy:

Number of user messages TODO allocation method Description Example
Single user >10 messages Exclusive TODO Users with many messages are analyzed separately @A (15 messages) → TODO: Analyze @A
Single user 5-10 messages 3-5 users merged into one TODO Users with medium message counts are analyzed in groups @B (7 messages) + @C (6 messages) + @D (5 messages) → TODO: Analyze @B/@C/@D
Single user <5 messages All users merged into one TODO Users with few messages are analyzed together @E (3 messages) + @F (2 messages) + @G (1 message) → TODO: Analyze @E/@F/@G

Decision logic(check in order, stop when satisfied):

  1. Iterate over all speakers and group users by message count:
    • High-output group: message count >10
    • Medium-output group: message count 5-10
    • Low-output group: message count <5
  2. Create TODOs:
    • High-output group: one independent TODO per user
    • Medium-output group: merge every 3-5 users into one TODO (e.g. 10 users split into 2-3 TODOs)
    • Low-output group: merge all users into 1 TODO

Use the tools supported by the current environment to manage TODOs (e.g. OpenCode's todowrite tool)

Step 2: Subagent instruction template

Each subagent receives the following instructions:

分析雪球时间线文件中指定用户的发言

**文件路径**: `<时间线文件绝对路径>`

**需分析的用户**：
- @用户 A：19 条发言
- @用户 B：6 条发言

**分析指引**：
1. 读取文件，定位目标用户的发言段落
2. 只总结本人观点（排除 `//` 或 `>` 开头的内容）
3. 提取：核心观点、涉及标的（$xxx$格式）、情绪倾向

**输出要求**：
直接返回分析结果（不要写文件），格式：

### @用户 A
- **发言数量**: 19 条
- **主要观点**:
  - 观点 1
  - 观点 2
- **涉及标的**: $XXX$, $YYY$
- **情绪倾向**: 乐观/中性/悲观

Step 3: Concurrent scheduling

  • Concurrency limit: no more than 3 subagents running at the same time
  • Execution workflow：
    1. Fill idle slots (start subagents)
    2. Wait for any subagent to complete
    3. Mark TODO as completed/failed
    4. Continue to the next batch

Step 4: Retry mechanism

  • Automatically retry when a subagent fails, up to 2 times
  • TODOs that keep failing are marked "⚠️ Analysis failed" in the report, and the user is prompted that they can analyze manually

Step 5: Result aggregation

Integrate the analysis results of all subagents to generate a complete investment analysis report:

  • Merge the analyses of all speakers
  • Identify the TOP3 market hot topics discussed across users
  • Comprehensive investment advice/risk warnings

Environment adaptation guide

The main agent should choose the appropriate method based on the tools available in the current environment:

  • OpenCode: use the task tool to start subagents, use todowrite to manage TODOs
  • Claude Code: use a method suitable for the current environment to start subagents
  • Core principle: each subagent's analysis result is returned directly through the context (no intermediate files are written)

Crawler Script Parameters

skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py [选项]
Parameter Description Default Value
--hours Crawl the last N hours 24
--days Crawl the last N days -
--start-date Start date (YYYY-MM-DD) -
--end-date End date (YYYY-MM-DD) Today
-o, --output Output filename Auto-generated

Note：--hours、--days、--start-date The three parameters are mutually exclusive.

Example

# 爬取最近 24 小时（默认）
skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py

# 爬取最近 2 小时
skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py --hours 2

# 爬取最近 7 天
skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py --days 7

# 指定日期范围
skills/crawl-xueqiu-my-timeline/scripts/crawl_xueqiu_home_timeline_api.py --start-date 2026-03-01 --end-date 2026-03-06

PDF Output Workflow

1. Save the analysis report

Save the analysis report as 雪球时间线_YYYYMMDD_YYYYMMDD.md, with the date being the start and end dates of the time range.

Example:雪球时间线_20260305_20260306.md

2. Convert to PDF

bunx mdpdf 雪球时间线_YYYYMMDD_YYYYMMDD.md \
  --style=skills/crawl-xueqiu-my-timeline/assets/github-markdown.css

For example:

bunx mdpdf 雪球时间线_20260305_20260306.md \
  --style=skills/crawl-xueqiu-my-timeline/assets/github-markdown.css

Optional Parameters：

  • --border=<size>: page margin (default 20mm), e.g. --border=15mm
  • --format=<format>: paper size (default A4), e.g. --format=A4
  • --orientation=<orientation>: orientation (default portrait)
  • --title=<title>: PDF title

3. Clean up temporary files

rm 雪球时间线_YYYYMMDD_YYYYMMDD.md

Complete Example

# AI 写入总结文件 → 雪球时间线_20260305_20260306.md

# 转换为 PDF（当前目录生成 PDF）
bunx mdpdf 雪球时间线_20260305_20260306.md \
  --style=skills/crawl-xueqiu-my-timeline/assets/github-markdown.css

# 删除临时 Markdown 文件
rm 雪球时间线_20260305_20260306.md

Result: generated in the current working directory 雪球时间线_20260305_20260306.pdf

Output Format

The home_timeline_*.md generated by the crawler script contains:

  • File header: time range, generation time, speech statistics table
  • Grouped by speaker: sorted in descending order by number of speeches
  • Within each group, in reverse chronological order: newer speeches come first
  • Each speech: publish time, content, quoted content, original link
  • Automatic filtering: official account market broadcasts

Notes

  1. You need to log in to your Xueqiu account first
  2. If you encounter a Verification page, you need to complete the verification manually and then run again
  3. The crawling process automatically handles pagination and the md5__1038 token
  4. Output files are saved in the current working directory
  5. Subagent Scheduling: analysis tasks are executed through subagents, with concurrency ≤ 3, and automatic retry twice on failure
  6. Environment Adaptation: the main agent selects the subagent startup method based on the current environment tools

Typical Use Cases

  1. Daily investment news review: crawl the last 24 hours, and AI summarizes market hotspots and generates a PDF report
  2. Follow big V updates: crawl the last N days, and AI analyzes changes in investment views and generates a PDF report
  3. Specific event tracking: specify a date range, and AI organizes the discussions during the event and generates a PDF report

Resource Files

  • assets/github-markdown.css: PDF style file (used by mdpdf)
  • scripts/check-cdp.sh: check Chrome Debug mode
  • scripts/check-agent-browser.sh: check agent-browser installation
  • scripts/crawl_xueqiu_home_timeline_api.py: crawler script (main entry)
  • evals/evals.json: test case set (used to verify skill functionality)