SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsxiaohongshu-crawler

xiaohongshu-crawler

General & other

View repository

Updated 10-16 00:24

About this skill

Xiaohongshu post scraping

📝 Overview

Scrape popular Xiaohongshu posts based on keywords, support filtering by date range and multiple sort methods (relevance, latest, hottest), and return note titles, authors, engagement data, etc., with results displayed in a structured table.

✨ Features

Feature Module Capability Description Core Value
Post Scraping Keyword search for Xiaohongshu posts Precisely discover high-heat content
Date Filtering Support filtering by date range Locate popular content in a specific time period
Multiple Sorting Options Three sorting options: relevance/latest/hottest Flexibly meet different filtering needs
Engagement Data Return saves, shares, comments, likes, etc. Comprehensively evaluate post popularity
Report Export CSV (Excel-compatible) + HTML visual report Local offline viewing and sharing

🔑 Authentication

  • Get API Key: Go to Red Fox Hub
  • Configuration method 1: Write to ~/.openclaw/openclaw.json → { "env": { "REDFOX_API_KEY": "ak_xxxx..." } }
  • Configuration method 2: Run in terminal export REDFOX_API_KEY="ak_xxxx..."

⚙️ Workflow

Step 1: 🔍 User Intent Understanding

Extract the following parameters from user input:

Parameter Extraction Rule Default Value
keyword Search keyword mentioned by the user Empty string ""
startDate Start date, parsed as YYYY-MM-DD 30 days ago (automatically calculated by script)
endDate End date, parsed as YYYY-MM-DD Today (automatically calculated by script)
sortType Sort method, see the table below "_0"

sortType enum:

Value Meaning Trigger Words
_0 Relevance sorting (default) Default sorting, comprehensive sorting, relevance
_2 Latest (sorted by publish time) Latest, newest, by time
_4 Hottest (sorted by engagement count) Hottest, most popular, hottest, by popularity

Time parsing rules:

User Input Parsing Method Example
Absolute date Parsed as YYYY-MM-DD 6月1号 → 2026-06-01
Relative date Calculated based on the current date 最近7天 → start=7 days ago, end=today
Date range Parse separately as startDate/endDate 5月30日到6月2日
No time provided Do not pass --start-date/--end-date, the script defaults to querying the last 30 days Do not pass parameters

Step 1.5: 🧠 Keyword Type Determination (⚠️ Must be executed before calling the script)

Core rule: Generic terms must be asked about before querying; specific terms can be queried directly.

Keyword classification:

Type Characteristics Handling method
Empty keyword The user did not provide a keyword (e.g., "recent popular notes") Directly query site-wide popular content, pass keyword as ""
Niche term Contains specific scenario/attribute/audience modifiers (e.g., "fat-loss meals", "petite styling", "keto diet") Call the script directly
Generic term Pure broad category term with no modifiers (e.g., "food", "styling", "beauty") Must first recommend niche terms, then wait for the user's reply before querying

Generic term handling process (⚠️ Directly calling the script is prohibited!):

Step 1: Generate 10 niche term recommendations

Generation principles:

  • Terms should be moderately sized, avoiding being too narrow (no data found) or too broad
  • Must cover different scenarios: 2-3 each of trend terms, audience terms, scenario terms, and intent terms

Output example:

I identified that "food" is a broad category and recommend the following niche directions: Home cooking, baked desserts, fat-loss meals, breakfast, late-night snacks, regional cuisines, lazy recipes, baby complementary foods, hot pot, coffee drinks Reply "expand" to search all 10 terms at once, or reply "do not expand" to continue searching "food"

Step 2: Wait for the user's reply (do not continue executing the script in the same conversation)

  • User replies "expand" → call the script to search the 10 niche terms (comma-separated)
  • User replies "do not expand" → call the script to search the original keyword

Step 2: 📡 Call the script

# 基础调用（仅关键词，日期默认最近30天，排序默认相关性）
python3 ~/.qoderwork/skills/xiaohongshu-crawler/scripts/crawl_xhs.py "<关键词>"

# 完整参数（仅传入非空的时间/排序参数）
python3 ~/.qoderwork/skills/xiaohongshu-crawler/scripts/crawl_xhs.py "<关键词>" --start-date <startDate> --end-date <endDate> --sort-type <_0|_2|_4>

The script returns JSON, including the articles array,total total count, and the following auxiliary fields:

Field Type Description
articles Array Work data (main displayed content)
relatedSearches Array Related search term recommendations
latestHotArticles Array Recent popular note recommendations (auxiliary content, displays 10 by default)
hotTopics Array Popular topic tags (for reference only, not displayed in the conversation)

Fields for each work (script internal fields ↔ raw API fields):

Script field Raw API field Description
title title Note title
author authorNickname Author nickname
collect_count collectedCount Favorite count
share_count sharedCount Share count
comment_count commentsCount Comment count
like_count likedCount Like count
publish_time createTime Publish time
work_url shareInfoLink Work link
cover cover Cover image URL
desc desc Note description/topic tags
author_fans authorFans Author follower count
interactive_count interactiveCount Total interactions
work_id id Work ID

Step 3: 📊 Result display

⚠️ Total count verification: The displayed N must be taken from the articles array length; manual counting is prohibited. ⚠️ Mandatory output rules: Directly read the JSON data returned by the script and output according to the corresponding strategy; adding extra analysis or suggestions is prohibited.

Case A: articles ≥ 20

A1. Output query scope:

📊 Keyword "XXX" crawled a total of N itemsXiaohongshu works, the detailed data is as follows:

A2. Output risk control notice (immediately after A1, before the table):

!!! Due to Xiaohongshu risk control rules, some work links may not redirect properly. You can copy the corresponding work title and search for it on Xiaohongshu to view it. Thank you for your understanding🙇‍♀️🙇‍♀️

A3. Display the first 20 rows in a table:

| # | 笔记标题 | 作者 | 收藏 | 分享 | 评论 | 点赞 | 发布时间 |
|---|---------|------|------|------|------|------|----------|
| 1 | [标题](work_url) | 作者名 | 1.2w | 3.5w | 8000 | 5.6w | 06-02 19:55 |

A4. Notice of remaining data:

The first 20 items of data are shown above, with M itemsnot displayed. Reply "View All" to expand the remaining data.

A5. Recommend niche tracks (generate 10 related niche direction words based on the current keyword):

If you want to explore a niche direction in depth, you can choose from the following keywords: {niche word 1}, {niche word 2}, ..., {niche word 10} Reply with a specific keyword and I will query it for you.

Case B: 0 < articles < 20

B1. Output query scope + low data notice:

📊 Keyword "XXX" crawled a total of N itemsXiaohongshu works. 💡 The current keyword has few results in this time period. You can try a shorter keyword or expand the time range.

B2. Output risk control notice:

!!! Due to Xiaohongshu risk control rules, some work links may not redirect properly. You can copy the corresponding work title and search for it on Xiaohongshu to view it. Thank you for your understanding🙇‍♀️🙇‍♀️

B3. Display the full table (same format as A3).

B4. Recommend niche tracks (same as A5).

Case C: articles = 0

😔 Sorry, no Xiaohongshu works related to "XXX" were found. Please try changing the keyword or adjusting the time range.

C1. Recommended search terms (extracted from the relatedSearches field, displayed in bold):

🔍 Recommended search terms: word 1, word 2, word 3...

⚠️ If relatedSearches is empty, this item will not be displayed.

C2. Recommend popular notes (take the first 10 from the latestHotArticles field; the table does not include the rating field):

💡 We have recommended recent popular notes for your reference:

| # | 笔记标题 | 作者 | 收藏 | 分享 | 评论 | 点赞 | 发布时间 |
|---|---------|------|------|------|------|------|----------|
| 1 | [标题](work_url) | 作者名 | 1.2w | 3.5w | 8000 | 5.6w | 06-02 19:55 |

⚠️ If latestHotArticles is empty, this block will not be displayed.

C3. Recommend popular tracks (extracted from the hotTopics field; if empty, use the following default list):

📈 You can also try searching the following popular tracks: Fashion, Food, Makeup, Film and TV, Workplace, Cute Pets, Home, Travel, Sports, Technology, Internet, Healthcare, Education, Parenting, Lifestyle

Formatting rules (applicable to all cases):

  • Numbers < 10000 are displayed directly; ≥ 10000 use the x.xw format
  • publish_time convert to MM-DD HH:MM; display regardless of whether a time range is specified
  • Titles over 30 characters are truncated with ..., using the [标题](work_url) link format
  • When the title is empty, display "-"

Step 4: 📦 More actions

After displaying the results, append the following content at the end:

⚡ 更多操作
• 是否需要下载 Excel 文件或 HTML 可视化报告？便于您在浏览器中打开查看
• 本次共 N 条作品，是否需要查看剩余 M 条？

⚠️ The second item is only displayed when articles > 20, N is the total count, M = N - 20.

4.1 Download report

Write the crawl result JSON to a temporary file, then call the report generation script:

echo '<JSON数据>' > /tmp/xhs_crawl_data.json
python3 ~/.qoderwork/skills/xiaohongshu-crawler/scripts/generate_report.py "<关键词>" \
  --input /tmp/xhs_crawl_data.json --format <csv|html|both>
--format Generate file Notify user example
csv CSV (Excel-compatible) ✅ Excel（CSV）文件已生成：~/Downloads/XhsCrawl/小红书作品_关键词_时间戳.csv
html HTML visualization report ✅ HTML 可视化报告已生成：~/Downloads/XhsCrawl/小红书作品_关键词_时间戳.html
both CSV + HTML Notify both file paths at the same time

4.2 User replies "View all"

Directly display the remaining data (starting from the 21st entry), using the same table format as A3, and no longer append the "More actions" prompt.