douyin-works-crawler

Douyin Content Scraping

Enter a Douyin name or ID to retrieve basic account information + recent content with one click

Overview

Douyin Content Scraping is an intelligent tool built specifically for Douyin content analysis, helping users quickly retrieve basic information and recent content data for any Douyin account.

By simply entering an account name or Douyin ID, you can:

📊 Retrieve basic account information (follower count, likes received, total content count, Honghu Index, etc.)
📋 View the recent content list (up to 50 items, including engagement data and content links)
🔍 Discover the top 3 content by engagement and get content analysis worth learning from

Suitable for brands, MCN agencies, content operations, self-media practitioners, and others who need to analyze Douyin account performance.

Features

🎯 Core Features

📊 Account Information Lookup: Enter a Douyin nickname or Douyin ID to retrieve basic account data with one click (follower count, likes received, total content count, Honghu Index, etc.)
📋 Recent Content Scraping: Automatically retrieve the recent content list (up to 50 items), including likes, comments, shares, engagement count, and content links
🔍 Data Highlights Analysis: Top 3 content by engagement analysis + account characteristic analysis (update frequency, content direction, engagement performance, viral content characteristics)

✨ Highlights

⚡ Smart Recognition: Automatically determines the input type (nickname/Douyin ID), no need to manually switch query modes
🔗 Direct Links: Nickname jumps to the account homepage, content list jumps to the video page
🔒 Safe and Reliable: API integration method, no need to log in to a Douyin account

One-Click Installation

Prerequisites

Python 3.6+
Honghu Data API key (format ak_xxx）

Installation Methods

Method 1: Direct Use (Recommended)

Ensure the project files have been downloaded locally

Configure environment variables:

# macOS/Linux
export REDFOX_API_KEY=你的API密钥值

# Windows PowerShell
$env:REDFOX_API_KEY="你的API密钥值"

Run the query:

python scripts/douyin_works_fetcher.py --account "抖音名称或抖音号"

Method 2: Configure on platforms such as Coze/Dify

Upload the skill folder to the platform
Configure in the environment variables REDFOX_API_KEY
Configure trigger words to invoke via conversation

Environment Variable Configuration

Variable Name	Required	Description
`REDFOX_API_KEY`	Yes	Honghu Data API key (format `ak_xxx`）

Usage Guide

Basic Usage

1. Query Account Content

Tell the assistant the Douyin account you want to query:

User: Scrape the Douyin content of "Zhou Yaogu Home Cooking"

Assistant: Retrieved the account data for "Zhou Yaogu Home Cooking" for you, 5.471 million followers, 47 recent content items...

2. Precise Query (Recommended)

Use the Douyin ID for precise queries to avoid fuzzy nickname matching:

User: Help me query the content of Douyin ID cdjjc028

Assistant: Account data for "Zhou Yaogu Home Cooking" has been precisely matched...

Advanced Usage

3. Export in JSON format

When structured data is needed, you can specify JSON output:

python scripts/douyin_works_fetcher.py --account "抖音号" --output json

Command Quick Reference

Command	Function
`爬取抖音作品 [名称/抖音号]`	Query account work data
`抖音作品列表 [名称/抖音号]`	Get recent work list
`抖音内容采集 [名称/抖音号]`	Collect account content
`导出抖音作品 [名称/抖音号]`	Export work data

Use Cases

Scenario 1: Brand Competitor Monitoring

Role: Brand Marketing Manager

Need: Monitor the content performance and engagement data of competitor Douyin accounts

How to Use：

Enter the competitor account's Douyin ID to query
View the recent work list and engagement data
Analyze the content characteristics of the top 3 works by engagement

Expected Benefits: Stay informed of competitor content trends in a timely manner and optimize your own content strategy

Scenario 2: MCN Agency Creator Evaluation

Role: MCN Operations Staff

Need: Evaluate the data performance and content direction of creator accounts

How to Use：

Query the target creator's basic account information (follower count, likes received, Honghu Index)
Analyze the engagement performance of recent works
View the account feature analysis to understand update frequency and content direction

Expected Benefits: Quickly evaluate creator value and assist signing decisions

Scenario 3: Self-Media Content Optimization

Role: Douyin Content Creator

Need: Learn the viral content characteristics of leading accounts in the same field

How to Use：

Query the work data of leading accounts in the same field
View the analysis of the top 3 works by engagement
Learn the points worth referencing from viral content

Expected Benefits: Find directions for content optimization and improve account engagement performance

Scenario 4: Data Analysis Report

Role: Data Analyst

Need: Batch-acquire structured data from Douyin accounts for analysis

How to Use：

Use the --output json parameter to export structured data
Batch-query multiple accounts
Combine with other data for comprehensive analysis

Expected Benefits: Efficiently acquire data to support analysis report output

Project Architecture

Directory Structure

douyin-works-crawler/
├── scripts/
│   └── douyin_works_fetcher.py   # 核心脚本（API调用+数据格式化）
├── references/
│   └── core_workflow.md          # 核心技能逻辑（接口规范、输出模板、处理规则）
├── CONFIG.json                   # 技能配置文件
└── SKILL.md                      # 技能说明文档

Tech Stack

Project	Description
Runtime Environment	Python 3.6+
Data Source	Honghu Data API
Authentication Method	API Key (X-API-KEY request header)
Output Format	Markdown / JSON

Core Module Description

DouyinWorksFetcher: Core class, encapsulating the two major functions of querying and data formatting
- query_account(): Query account information and work list
- format_markdown() / format_json(): Output formatting

📌 For the complete interface specifications, output templates, processing rules, and other core logic, see references/core_workflow.md, and the Agent must follow all rules in that file when executing.

FAQ

Installation-related questions

Q1: What should I do when the runtime prompts “Environment variable REDFOX_API_KEY is not set”?

A: Please configure the environment variable first:

# macOS/Linux
export REDFOX_API_KEY=你的API密钥值

# Windows PowerShell
$env:REDFOX_API_KEY="你的API密钥值"

Q2: How do I obtain the RedFox Data API key?

A: Go to the RedFox platform to register and apply for an API key. The format is ak_xxx。

Usage-related questions

Q3: Why is Chinese nickname lookup not supported?

A: Chinese nicknames are not unique (multiple accounts may use the same nickname), and the broad-domain database has a low inclusion rate for nicknames. Please use the Douyin ID for lookup. How to view the Douyin ID: Douyin APP → target account homepage → below the nickname.

Q4: Why can some accounts not be found?

A: The account may currently have no data records. It is recommended to use the Douyin ID for precise lookup, or try again later.

Q5: What is the maximum number of items displayed in the works list?

A: Recent works data is limited to a maximum of 50 items, sorted in reverse chronological order by publish time.awemeCount The field is the total number of historical works for the account; the number in the works list may be less than this value.

Troubleshooting

Q6: What should I do when an API call returns the error “Insufficient credits”?

A: The RedFox API is billed by number of calls. Please go to the RedFox platform to top up credits.

Q7: What should I do if the query times out?

A: Please check whether the network connection is normal. The script default timeout is 30 seconds. If timeouts persist, try again later.

📌 Trigger scenarios

"Crawl Douyin works", "Douyin works list", "View Douyin videos"
"Douyin content collection", "Scrape Douyin works", "Export Douyin works"
"Download Douyin works data", "Douyin works data export"
Directly enter Douyin name/ID + "crawl/collect/export works"

🔌 Data source

Only data source: RedFox Data API

Query interface

Configuration item	Value
Interface address	`POST https://redfox.hk/story/api/dy/data/listWorkByAccount`
Authentication	Request header `X-API-KEY`(environment variable `REDFOX_API_KEY`, format `ak_xxx`）
Credits	Yes (resourceId: `/story/api/dy/data/listWorkByAccount`）

For detailed interface parameters, response fields, and data range descriptions, see references/core_workflow.md

🎯 Core features

Feature 1: Account basic information

Query and display the account's core basic data:

Field	Description	Example
Nickname	Account display name (clickable to jump to the Douyin homepage)	Dong Yuhui
Douyin ID	Account unique identifier (authorUniqueId)	dongyuhui
UID	Platform internal ID (authorUid)	100476191179
Follower count	Total followers (authorFansCount, ≥10,000 uses w, ≥100 million uses 亿)	2586.3w
Total works	Historical post count (pagination total)	561

Feature 2: Recent works list

Display recent works data, supports pagination (maximum 50 items/page):

Field	Description
Work title	content field (work body/description, first 50 characters)
Publish time	publishTime
Like count	likeCount
Comment count	commentCount
Shares	shareCount
Favorites	collectCount (new field added in the new API)
Interactions	Manual calculation: likes + comments + shares + favorites
Work link	opusUrl (direct link to the work)

Feature 3: Data Highlights Analysis

Automatically analyze and output:

Top 3 by Interactions: The top 3 works sorted by interaction count, including an interaction structure analysis for each work (share rate, comment rate, fan interaction ratio, etc.) and content feature analysis
Account Feature Analysis: Update frequency, content direction, interaction performance, viral hit characteristics

📝 Output Template

Usage Instructions: Strictly follow the template for output; sections must not be omitted. Data comes from the API and must be presented as-is.

Number Format Rules: ≥10,000 use w(e.g., 3.2w), ≥100 million use 亿(e.g., 1.5亿), <10,000 use comma thousands separators (e.g., 8,642).

Link Column Format: When opusUrl is available, display [链接](opusUrl), when opusUrl is not available, display -。

Nickname Link Format: The nickname field uses the [nickname](https://www.douyin.com/user/{secUid}) format; when secUid is empty, display plain text.

Interaction Count Calculation: The new API has no interactiveCount field, so it must be calculated manually: likes + comments + shares + favorites.

## 🎬 [nickname] - 抖音作品数据

### 账号基础信息

| 昵称 | 抖音号 | UID | 粉丝数 | 作品总数 |
|------|--------|-----|--------|---------|
| [nickname](https://www.douyin.com/user/[secUid]) | [accountId] | [uid] | [followerCount] | [total] |

---

### 近期作品（共[N]条）

| # | 发布时间 | 标题 | 点赞 | 评论 | 分享 | 收藏 | 互动 | 链接 |
|---|---------|------|------|------|------|------|------|------|
| 1 | [publishTime] | [content] | [likeCount] | [commentCount] | [shareCount] | [collectCount] | [互动合计] | [链接](opusUrl) / - |

> 作品列表为近期作品数据，支持分页（最多50条/页），账号作品总记录数为 [total] 条。

---

### 数据亮点

#### 互动量TOP3

🥇 **[title]**（互动[interactiveCount]）
> [根据账号数据、作品内容、内容定位等总结该作品值得学习点]

🥈 **[title]**（互动[interactiveCount]）
> [根据账号数据、作品内容、内容定位等总结该作品值得学习点]

🥉 **[title]**（互动[interactiveCount]）
> [根据账号数据、作品内容、内容定位等总结该作品值得学习点]

#### 账号特征分析

- **更新频率：** [基于近期作品发布间隔分析]
- **内容方向：** [基于作品标题关键词归纳内容赛道]
- **互动表现：** [基于互动数据整体趋势分析]
- **爆款特征：** [基于TOP3作品共性总结]

---

### 数据说明

- **数据范围：** 近期作品数据，支持分页（最多50条/页），按发布时间倒序
- **互动数：** 点赞+评论+分享+收藏
- **作品链接：** 接口返回opusUrl字段，提供作品直达链接
- **数据来源：** 红狐数据API（广域库）

*爬取时间：[YYYY-MM-DD HH:mm]*
> 💼 另外红狐配套全量数据库可提供完整详实数据，如需了解采购方案，可前往红狐hub[企业服务](https://redfox.hk/dashboard/enterprise)对接咨询

🔍 Query Miss Handling

When the query does not find the target account,online searching is strictly prohibited, handle it according to the following process:

Chinese nickname input → block directly, output a guidance prompt (do not initiate an API call), and guide the user to provide the Douyin ID
Matched the wrong account → prompt: "The nickname query returned "[actual matched nickname]", which is not the account you are looking for. Please provide the target account'sDouyin IDfor precise querying."
Account not found → output prompt message:

未查询到当前账号的相关信息，请检查输入的抖音号或昵称是否正确。
建议使用抖音号进行精准查询，或稍后再试。

Fallback Strategy

When an API call fails (such as insufficient credits or network abnormality), clearly inform the user of the error reason,online searching is strictly prohibited, and estimating or supplementing fields not returned by the API from third-party channels is strictly prohibited。

🔧 Usage

Script Invocation

Query account works:

python3 scripts/douyin_works_fetcher.py --account "抖音名称或抖音号"

Parameter Description

Parameter	Description
`--account`	Douyin nickname or Douyin ID (auto-detect: contains Chinese → nickname, non-Chinese → Douyin ID)
`--output`	Output format:`markdown`(default) / `json`

Auto-detection Logic

All inputs uniformly use the uniqueName parameter for querying. Chinese nicknames are not unique identifiers; when Chinese input is detected, block directly and guide the user to provide the Douyin ID (supports three types: pure letters, letters + numbers, pure numbers).

⚠️ Notes

🔌 Single data source: All data is obtained only from the Honghu API; third-party channels are not used to supplement or estimate
📊 Data scope: Recent work data, supports pagination (up to 50 items/page); the broad-domain library only includes popular data
🔢 Number format: ≥10,000 is displayed as x.xw(e.g., 3.2w), ≥100 million displayed as x.x亿, <10,000 uses thousands separators (e.g., 8,642)
🔗 Work link: The API returns the opusUrl field, displayed in the table as [链接](opusUrl), when there is no opusUrl, display -
🧮 Interaction count: The new API has no interactiveCount field, it must be calculated manually (likes + comments + shares + favorites)
👤 Account information: Extracted from the work item's author* fields, there is no separate account information API
❌ Chinese nickname query: When Chinese input is detected, block directly without making an API call, and guide the user to provide a Douyin ID (letters only/letters+numbers/numbers only)
❌ Account not found: When the API returns empty data, output a prompt message guiding the user to use a Douyin ID for precise query,Online search is strictly prohibited, generating reports without basis is strictly prohibited
❌ Nickname matched to the wrong account: Prompt the user that the current match is not the target account, and guide them to provide a Douyin ID to query again,Online search is strictly prohibited
✅ Output truthfully: The number of works is displayed truthfully

📌 The complete API specifications, output templates, processing rules, and other core logic are detailed in references/core_workflow.md, Agent execution must follow all rules in this file.

🛠️ Version Information

Version number: v3.0
v3.0 update: Switched to /story/api/dy/data/listWorkByAccount Wide-area database API; request parameters uniformly use uniqueName(both Douyin ID and nickname use this parameter, userId is no longer distinguished); response structure changed (works are in data.list, account information is extracted from author* fields); added collectCount field; interaction count changed to manual calculation (likes + comments + shares + favorites); work link field changed to opusUrl; pagination supported (pageNum/pageSize); added time range filtering (startDate/endDate); removed region/likes/Red Fox index fields (not returned by the new API)
v2.3 update: Output a friendly prompt message when no account is found; account feature analysis adds a content direction dimension; added the single-call principle to avoid wasting credits
v2.2 update: Number format specification (10,000→w, ≥100 million uses 亿, <10,000 uses thousands separators); link column format fixed as[链接](url)/-
v2.1 update: The API added the url field, supporting direct work link output
v2.0 update: Switched to /dyData/queryUserWithWorks API, supports crawling of recent work data (up to 50 items); request parameters changed to single values (accountId/accountName); success code changed to 2000; work fields updated (workList/likeCount/publishTime)
v1.0 update: Initial version, based on the Red Fox API /story/api/dyUser/query to implement account information + last 7 days of work crawling

About this skill

Overview

Features

🎯 Core Features

✨ Highlights

One-Click Installation

Prerequisites

Installation Methods

Method 1: Direct Use (Recommended)

Method 2: Configure on platforms such as Coze/Dify

Environment Variable Configuration

Usage Guide

Basic Usage

1. Query Account Content

2. Precise Query (Recommended)

Advanced Usage

3. Export in JSON format

Command Quick Reference

Use Cases

Scenario 1: Brand Competitor Monitoring

Scenario 2: MCN Agency Creator Evaluation

Scenario 3: Self-Media Content Optimization

Scenario 4: Data Analysis Report

Project Architecture

Directory Structure

Tech Stack

Core Module Description

FAQ

Installation-related questions

Usage-related questions

Troubleshooting

📌 Trigger scenarios

🔌 Data source

Query interface

🎯 Core features

Feature 1: Account basic information

Feature 2: Recent works list

Feature 3: Data Highlights Analysis

📝 Output Template

🔍 Query Miss Handling

Fallback Strategy

🔧 Usage

Script Invocation

Parameter Description

Auto-detection Logic

⚠️ Notes

🛠️ Version Information