SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsdouyin-transcript-exporter

douyin-transcript-exporter

Data & analytics

View repository

Updated 10-15 23:32

About this skill

Douyin Verbatim Transcript Export

Execution Environment

Must be executed in Doubao Work, and use its built-in verbatim transcript retrieval capability. When public information from a Douyin homepage needs to be collected, you can log in to Douyin through Doubao Work's built-in browser; it is recommended to use a dedicated alt account. This login is only used to read video information from public homepages, not to download videos, and does not affect verbatim transcript retrieval.

Input

Input Item Required Description
Douyin creator homepage URL or single video URL Yes Creator homepage https://www.douyin.com/user/xxxxx, or single video https://www.douyin.com/video/xxxxx, or Douyin share short link
Feishu Bitable URL No wiki link or base link, including the table parameter.If the user does not provide it, you must proactively ask; if the user explicitly says not to use a table, save to a local JSON/CSV file
Number of videos No Defaults to the latest 5; when the user says "latest N", collect only the first N
Whether to extract verbatim transcripts No Extracted by default; skip when the user says "only basic data"

Collection Fields (Standard 9 Columns + AI Extension 2 Columns)

Basic collection fields (9 columns, scraped directly from the Douyin page): Account name (single select), title, introduction, verbatim transcript, likes, comments, shares, publication date, video link

AI extension fields (2 columns, generated by AI based on the verbatim transcript):

  • Topic direction (multi-select select): choose 1-3 from preset tags, such as AI collaboration methods, AI beginner tutorials, enterprise AI implementation, cognitive thinking, prompt engineering, personal growth, productivity tools, digital transformation, learning methods, AI management, etc. If the target table already has options for this field, you must choose from the existing options and must not add new ones yourself.
  • Topic summary (text): use one sentence (30-80 characters) to summarize the video's core viewpoint or main content, requiring precision, no repetition of the title, and distillation of the essence.

Skip the AI extension fields when the user says "only basic data"; skip when the target table does not have these two fields (do not automatically create extension fields unless the user explicitly requests it).

Execution Flow

Step 0: Pre-checks and inquiries (must be executed first)

Before starting any collection operation, you must check whether the user has provided the following necessary information; when missing,you must proactively ask the user, and must not assume or skip on your own:

  1. Check the Douyin source address：

    • Has the user provided a creator homepage URL, a single video URL, or a Douyin share short link?
    • When not provided: ask the user "Which Douyin creator's videos do you want to collect? Please provide the creator homepage link or video link."
    • When a share short link is provided (such as https://v.douyin.com/xxxxx/), first open the short link in the browser to obtain the real URL after redirection (creator homepage or video detail page).

  2. Check the Feishu Bitable address：

    • Has the user provided a Feishu Bitable URL (wiki link or base link)?
    • When not provided: ask the user "Which Feishu Bitable should the collection results be written to? Please provide the table link. If you don't need to write to a table, I can save to a local file."
    • The user explicitly states no table / save locally: skip the branch, and after collection is complete save the results as local JSON and CSV files (see the local save branch in Step 4).
    • The user provided a table address: continue the normal flow and write to the Feishu table.

  3. Confirm other optional parameters：

    • Number of videos: when the user says "latest N", collect only the first N; if not specified, default to the latest 5.
    • Whether to extract transcripts: skip when the user says "only basic data"; if not specified, extract by default.

  4. Start only after the information is complete: only after the above required information is confirmed complete may you proceed to Step 1 environment detection. Do not start collection when the information is incomplete.

Step 1: Environment detection

  1. Detect lark-cli identity: run lark-cli api GET /open-apis/authen/v1/user_info --as user, and confirm that the logged-in account is the user's own account (not test accounts such as "来点羊蝎子" that come with the sandbox).

    • If the sandbox lark-cli has no permission, try the lark-cli in the system path: on Mac /opt/homebrew/bin/lark-cli, on Windows where lark-cli to locate it.
    • If neither identity has permission, tell the user they need to first authorize in the Feishu document or switch the lark-cli login account.

  2. Detect the operating system: on Mac use mac_computer_use_tool(plane="bu"), on Windows use computer_use_tool(plane="bu"). The code logic for both is completely identical, both are import seed_browser_use as bu。

Step 2: Douyin video collection

Read references/douyin.md to get the collection details. Core steps:

  1. Open the creator's homepage in a browser and check whether Douyin is already logged in.

    • When not logged in (the page shows a login popup, or the works list shows "service exception"), call interaction.request_action(type="browserControl") to request the user to take over and complete login.
    • After login, wait for the works list to finish loading.

  2. Scroll the page to load all videos, and extract all video links from the DOM (/video/xxxxx format) and basic information.

    • The title, caption, and like count of each video can be obtained directly from the homepage text.
    • When the user specifies "latest N", take only the first N.

  3. Enter each video detail page one by one (https://www.douyin.com/video/xxxxx), and extract:

    • Comment count, share count, favorite count (the numbers in the page interaction area, order: like → comment → favorite → share)
    • Publish date ("发布时间：YYYY-MM-DD HH:MM" at the bottom of the page)
    • Creator nickname (author information at the top of the page)

  4. Aggregate into a structured data list, each entry containing: video_id, url, title, description, likes, comments, shares, publish_date, author_name.

Step 3: Transcript extraction (executed by default)

Call web.fetchfor each video link, and the Douyin page will return the video's complete spoken transcript.

[Mandatory requirement] The complete verbatim transcript must be obtained; writing any placeholder explanation (such as "complete content obtained via web.fetch") is prohibited. If retrieval fails, leave the field empty and note the reason for failure; never use placeholder text to pass it off as complete content.

  1. Call web.fetch: for the first call use pagination.offset=0, and from the returned result check pagination_content 's end_offset and total_length。

    • If end_offset < total_length, it means the content was truncated,you must continue reading paginated results: for the next call set pagination.offset = 上一次的 end_offset, and repeat until end_offset >= total_length。
    • Concatenate the content returned by all pages into the complete text.

  2. Separate the verbatim transcript body: separate the title/copy part and the verbatim transcript body part from the complete content (the verbatim transcript body is usually after the first blank line).

  3. Content completeness check(if any condition is met, it is considered incomplete and you must fall back):

    • Verbatim transcript body length < 50 characters (short-video voiceovers are usually at least several dozen characters)
    • The content suddenly cuts off in the middle of a sentence (the end is not a period/question mark/exclamation mark/ellipsis or other ending punctuation)
    • The returned content contains placeholder text such as "complete content", "web.fetch", "obtain", etc.
    • The web.fetch call fails or returns empty content

  4. Fallback path: when the content is incomplete, fall back to the Feishu Minutes pipeline: first load the doubao-video-extract skill, confirm the script path, then run python3 scripts/minutes/social_video_to_minutes.py "<url>" --run-lark。

    • This method will download the video audio → upload it to Feishu Minutes → transcribe, which takes a long time (1-3 minutes per item).
    • If the fallback still fails, leave the verbatim transcript field empty and note in the result summary that the verbatim transcript retrieval for that item failed.

  5. When the user explicitly says "no verbatim transcript", skip this step and leave the verbatim transcript field empty.

Step 3.5: AI generates topic directions and theme summaries (executed by default, requires the target table to contain the corresponding fields)

Based on each video'stitle + description + verbatim transcript, AI generates two extended fields in batches:

  1. Topic direction (multiple choice)：

    • First use lark-cli base +field-list to read the existing option list of the target table's "Topic direction" field.
    • You must select 1-3 of the most fitting tags from the existing options,adding new options on your own is prohibited。
    • Selection criteria: which domain the video's core topic falls into; prioritize the single most important one, then choose 1-2 related ones as secondary choices.
    • If the target table has no "Topic direction" field, skip this field.

  2. Theme summary (text)：

    • Use one sentence (30-80 characters) to summarize the video's core viewpoint or main content.
    • Requirements: accurately distill the essence, do not repeat the original title, and do not start with filler such as "this video talks about...".
    • If the video is purely sharing/unboxing/lifestyle content, summarize its core information or feelings.

  3. Execution method：

    • Organize all videos' titles, descriptions, and verbatim transcripts into a structured list and hand them to AI for batch generation at once, avoiding item-by-item calls that waste context.
    • Transcript truncation strategy: For short videos (transcript <1500 characters), use the complete content; for long videos (transcript >1500 characters), truncate to the first 2000 characters (rather than 1000 characters) to ensure the video's core viewpoints are covered. If the video content has a lot of preamble at the beginning, the truncation length may be appropriately increased.
    • The output format is JSON:[{"video_id":"xxx","选题方向":["标签A","标签B"],"主题总结":"..."},...]
    • After generation, merge with the original data and prepare to write.

When the user says "only basic data is needed" or the target table does not contain these two fields, skip this step.

Step 4: Write results (Feishu table / local file)

Based on the user's choice in Step 0, it is divided into two branches:

Branch A: Write to Feishu multidimensional table (when the user provides a table address)

Read references/lark-write.md to obtain write details. Core steps:

  1. Parse the table link：lark-cli base +url-resolve --url "<表格URL>" --as user, and obtain base_token, table_id, view_id.

    • A wiki link can also be parsed directly, without manual conversion.

  2. Compare fields：lark-cli base +field-list --base-token <token> --table-id <id> --as user, and check whether the target table contains all 9 basic fields.

    • Automatically create missing basic fields: for account name use select(single select, multiple=false), for title/introduction/transcript/video link use text, for likes/comments/shares use number, for publication date use datetime。
    • Use an array format for batch field creation:--json '[{"name":"xxx","type":"xxx"},...]'。
    • AI extended fields (topic direction, theme summary): if they already exist in the target table, include them in the write; if they do not exist, thendo not automatically create them(unless the user explicitly requests it), and directly skip writing these two fields.
    • If the "topic direction" field exists, its existing option list must first be read, and when AI generates, it must strictly select from the existing options.

  3. Incremental deduplication: query the existing "video link" field value list in the table, compare it with the video links collected this time, and only write new records that do not exist.

    • Use lark-cli base +record-list or +record-search to obtain existing records.
    • When the user explicitly requests "full overwrite", skip deduplication and append directly.

  4. Batch write: use lark-cli base +record-batch-create to write new records all at once (maximum 200 records per batch).

    • For date fields, use "YYYY-MM-DD HH:MM" string format, or a millisecond timestamp.
    • For single-select fields (account name), use ["选项名"] array format.
    • After writing, return record_id_list to confirm the number of successful writes.

  5. Data quality verification(must be executed after writing):

    • Use lark-cli base +record-list to read the records newly written this time, and check the following quality issues:
      • Placeholder check: whether the transcript field contains placeholder text such as "complete content", "web.fetch", "passed", etc. (if so, it indicates the content was incomplete at write time and must be re-obtained and updated)
      • Length check: whether the transcript field is too short (<50 characters and not a pure music/no-speech video), and whether the title/introduction fields are empty
      • Key field check: whether the video link, publication date, and account name are written correctly
    • When a quality issue is found, immediately re-obtain that record's data and use +record-batch-update Update the corresponding record until verification passes.

Branch B: Save to local files (when the user explicitly states tables are not needed)

When the user explicitly states in Step 0 that Feishu tables are not needed, save the collection results as local files, using theseparate directory per batch + separate Markdown file per videostructure:

  1. Batch directory: Create in the current working directory a douyin_data/ folder, with a separate subdirectory created for each batch collection.

    • Directory naming:{博主昵称}_{采集日期YYYYMMDD}_{视频数量}条/, e.g. 瑶瑶_20260827_50条/
    • If the blogger's nickname contains special characters, replace them with underscores or remove the special characters.

  2. Separate Markdown file per video: In the directory, create for each video a .md file.

    • File naming:{序号}_{视频ID}.md, e.g. 01_7655189977860862854.md、02_7654993145986808761.md
    • The sequence number starts from 01 in collection order (newest first); use 1 digit for fewer than 10 items, and 3 digits for more than 99 items.
    • Markdown file content format: 
      # {视频标题}

## 基本信息
- **账号名称**：{博主昵称}
- **发表日期**：{YYYY-MM-DD HH:MM}
- **视频链接**：{url}
- **点赞**：{点赞数}
- **评论**：{评论数}
- **转发**：{转发数}

## 介绍/文案
{视频介绍或文案全文}

## 选题方向
{标签1}、{标签2}、{标签3}

## 主题总结
{一句话主题总结}

## 逐字稿
{完整口播逐字稿全文}

  3. Summary file (optional): In the batch directory, also save a summary _all.json file, containing the structured data of all videos for subsequent program processing.

    • The JSON format is an array, with each entry containing all fields (video_id, url, account name, title, description, transcript, likes, comments, shares, publication date, topic direction, theme summary).

  4. Data quality verification(must be performed after saving):

    • Check whether the number of Markdown files in the batch directory matches the expected collection count.
    • Read 2-3 Markdown files as samples and check:
      • Placeholder check: Whether the transcript section contains placeholder text such as "complete content", "web.fetch", "passed", etc.
      • Length check: Whether the transcript is too short (<50 characters and not a pure music video), and whether the title/description is empty
      • Format check: Whether the Markdown structure is complete (including sections such as basic information, description, topic direction, theme summary, transcript, etc.)
    • When problems are found, re-fetch that data entry and update the corresponding Markdown file until verification passes.

  5. Deliver files: Use the present_files tool to deliver the batch directory to the user, explaining the directory path, number of videos, and file structure (one Markdown file per video).

Result summary (common to both branches)

Report the collection results to the user:

  • Total number of videos collected
  • Write method: Feishu table (provide the table link) / local file (provide the file path)
  • Feishu table branch: number of new writes, number of duplicates skipped
  • Number of transcript retrieval failures (if any)
  • Data quality verification results (passed / number of issues fixed)

Key notes

  • Pre-check (must be performed first): At the beginning, you must check whether the user has provided a Douyin address (blogger homepage or video link) and a Feishu spreadsheet address. If the Douyin address is missing, you must ask; if the spreadsheet address is missing, you must ask, and if the user explicitly says no spreadsheet is needed, save locally (a separate directory for each batch, one Markdown file per video). Do not start collection when the information is incomplete.
  • Douyin share short link handling: When the user provides a https://v.douyin.com/xxxxx/ short link, first open it in a browser to obtain the real URL after redirection, then proceed with collection.
  • Douyin anti-scraping: Collection must be done through a real browser; it is forbidden to directly request the Douyin API with curl/requests. Login state is a prerequisite for obtaining complete data.
  • lark-cli path: The lark-cli in the sandbox environment may be logged in to a test account; prefer the lark-cli installed on the user's system. Default Mac path /opt/homebrew/bin/lark-cli。
  • Primary field restriction: The first field (primary field) of a Feishu Bitable cannot be deleted, only renamed. If the target table's primary field is the default "Text" and is empty, it should be renamed to "Title" rather than deleted.
  • Field type conversion: Type conversions such as text to select may require first confirming data compatibility; after conversion, the original values will automatically be mapped to options (if they match).
  • Transcript completeness (highest priority): A single transcript may be very long (thousands of characters). The Feishu text field supports large text, so no splitting is needed.You must ensure that the complete transcript is written; it is forbidden to write any placeholder explanation (such as "the complete content has been obtained via web.fetch"). When the content returned by web.fetch is truncated, you must read it in pages until it is complete.
  • Data quality verification: After writing, you must check whether there are placeholders, whether the transcript length is reasonable, and whether key fields are empty; fix any problems immediately.
  • AI extended field generation: The topic direction must be selected from the existing options in the target table; it is forbidden to add new options on your own; the topic summary should be controlled to 30-80 characters, distilling the essence without repeating the title. During batch generation, use the complete transcript for short videos, and for long videos it may be truncated to the first 2000 characters (rather than 1000 characters) to ensure coverage of the core viewpoints.

Reference documents

  • references/douyin.md: Specific selectors, data formats, login handling, and common exceptions for Douyin collection
  • references/lark-write.md: Feishu spreadsheet field creation, record writing, deduplication logic, CellValue format