SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillslive-transcript-summary

live-transcript-summary

General & other

View repository

Updated 10-16 09:52

About this skill

Summarize While Listening Skill

Monitor the audio recording transcription in the current WPS note in real time, polling once every 60 seconds,only process newly added sentences (incremental mode), automatically select a template based on the scenario to organize the content and write it back to the note.

Tool Priority

Prefer using wpsnote-cli(CLI), and fall back to MCP tool calls only when it is not supported.

Operation CLI Command MCP Tool
Get current note wpsnote-cli current --json get_current_note()
Get outline wpsnote-cli outline --note_id ID --json get_note_outline()
Get transcription wpsnote-cli audio --shorthand_id ID --json get_audio_transcript()
Search notes wpsnote-cli find --keyword KW --json search_notes()
Search content within notes wpsnote-cli search --note_id ID --query Q --json search_note_content()
Insert content wpsnote-cli edit --json-args '...' edit_block(op="insert")
Replace content wpsnote-cli edit --json-args '...' edit_block(op="replace")
Batch edit wpsnote-cli batch-edit --json-args '...' batch_edit()

When XML in the CLI contains angle brackets,uniformly use --json-args to pass a JSON object, to avoid shell escaping issues.

⚠️ Mandatory rules when writing to notes

content The parameter must be an XML string, and must never be an array or object.

// ✅ 正确
{ "content": "<h2>标题</h2><p>内容</p>" }

// ❌ 禁止——数组
{ "content": [{"type": "text", "text": "<h2>标题</h2>"}] }

// ❌ 禁止——对象
{ "content": {"type": "text", "text": "<h2>标题</h2>"} }

When encountering BLOCK_NOT_FOUND or repeated write failures,the first step is to check whether content is a string, then use outline to refresh the block_id and retry.

Core Workflow

启动（仅一次）:
  1. 检查 CLI 可用性
  2. 获取当前笔记 note_id
  3. 风格学习（搜索 3-5 篇历史同类笔记）
  4. 初始化状态文件
  5. 输出启动告知

loop（每 60 秒）:
  1. 扫描笔记大纲，找 NoteAudioCard
  2. 全量拉取转写，过滤新增句子（start_time > last_end_time）
  3. 若无新增 → 检测临时区用户输入 → 更新状态 → sleep 60 → continue
  4. 首轮：场景识别 + 发言人推断
  5. 人名联动：提取人名 → 搜索笔记库 → 注入背景上下文
  6. 首轮：先输出内容地图，再开始写入
  7. 用新增句子生成摘要 XML
  8. 写回笔记（首轮在末尾插入；后续追加到 summary_anchor 之后，始终在临时区之前）
  9. 回扫大纲：核查新内容是否落在正确章节，发现位置错误立即修正
 10. 首轮：在笔记末尾创建临时区（hr + 提示块）
 11. 检测临时区用户输入 → 合并到正文 → 清空用户 block
 12. 检测 *** 补全请求
 13. 更新状态文件（last_end_time、summary_anchor_id、temp_zone_anchor_id 等）
 14. 输出本轮简报 → sleep 60 → loop

Startup Phase

Step 1: Check CLI availability

wpsnote-cli status
# 输出包含"连接中... 成功" → CLI 可用，否则退回 MCP 模式

Step 2: Get the current note

NOTE=$(wpsnote-cli current --json)
NOTE_ID=$(echo $NOTE | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['data']['note_id'])")

# MCP 备选
get_current_note() → { note_id, title }

If it returns NO_ACTIVE_EDITOR_WINDOW, prompt the user to open the note and retry.

Step 3: Style learning

Search 3-5historical notes of the same type (meeting minutes/class notes/summaries), reading at most 3000 charactersper note, and comprehensively summarize the style characteristics:

NOTES=$(wpsnote-cli find --keyword "会议纪要 纪要 会议总结 周会 月会" --limit 5 --json)

Summary dimensions(take the majority by voting across multiple notes): language style (spoken vs written), structural preference (bullet list vs sectioned chapters), level of detail, whether Chinese and English are mixed, whether color blocks/columns are commonly used.

If there are fewer than 3 samples, continue and note "insufficient style samples, limited reference".

Step 4: Initialize the state file

state = {
    "note_id": note_id,
    "shorthand_id": None,        # 首轮扫描到 AudioCard 后填入
    "scene": None,               # 首轮场景识别后填入
    "last_end_time": -1.0,       # -1 代表从头开始
    "summary_anchor_id": None,   # 上轮最后写入的 block_id（临时区 hr 之前）
    "temp_zone_anchor_id": None, # 临时区 highlightBlock 的 block_id
    "round": 0,
    "speaker_map": {},           # {"发言人 1": "真实姓名"}
    "person_cache": {},          # {"张总": [{"note_title": ..., "snippet": ...}]}
    "started_at": time.time(),
}
# 保存到 /tmp/lts_state_{note_id}.json

Step 5: Output the startup notice

已开始边听边总结 ✓

当前笔记：[笔记标题]
识别场景：等待录音开始后自动识别
参考风格：[找到 N 篇 / 未找到历史总结]
模式：增量（只处理新增转写）+ 人名联动
轮询间隔：60 秒

---
援助功能：
  ① 补全：在笔记任意位置输入 ***，AI 结合录音自动补全
  ② 临时区：笔记末尾灰色区域，写下想法，AI 下轮自动合并到正文
  ③ 停止：告诉我「停止总结」即可
---

Polling Main Loop

Step 1: Scan audio cards

outline_data = json.loads(wpsnote_cli_outline(note_id))
audio_blocks = [b for b in outline_data["data"]["blocks"] if b["type"] == "note_audio_card"]

if not audio_blocks:
    # sleep 60, continue
    pass

audio_block = audio_blocks[0]  # 取第一个，或 status=recording 的那个
shorthand_id = audio_block["attrs"]["shorthand_id"]
status = audio_block["attrs"].get("status", "unknown")
# recording / paused / stopped 均继续处理

Step 2: Incrementally get the transcription

all_sentences = get_transcript(shorthand_id)  # 每个 sentence: {text, speaker, start_time, end_time}

# 只取新增句子
new_sentences = [s for s in all_sentences if s["start_time"] > state["last_end_time"]]

if not new_sentences:
    # → 跳到 Step 8（检测临时区），然后 sleep 60
    pass

wpsnote-cli audio only supports full retrieval; filtering is done locally, and the actual processing volume per round is constant.

Step 3: Scenario recognition (first round)

if state["scene"] is None and new_sentences:
    sample = " ".join(s["text"] for s in new_sentences[:10])
    state["scene"] = detect_scene(note_title, sample)  # 见「场景识别表」

Step 4: Speaker identity inference

for s in new_sentences:
    if "我是" in s["text"] or "我叫" in s["text"]:
        state["speaker_map"][s["speaker"]] = extracted_name
        # 若发现纠正，批量替换笔记中旧名称

Step 5: Name linkage (cross-note background injection)

Extract names from newly added sentences and speaker_map, search the note library, and inject the background context into this round's summary prompt.

Search strategy(degrade in order): full name → last two characters → single character of the name (use with caution)

Take at most 2 notes × 3 paragraphs per person, cache the results in state["person_cache"], and do not search repeatedly within the same session.

For detailed code, see person-linking.md

Step 6: First-round output content map

Triggered only in the first round (state["round"] == 0): Before writing notes, organize the overall logical structure based on the full transcript, present it to the user as a tree diagram, then start writing directly.

📋 内容地图

[笔记标题]
├── 基本信息（时间、参与人等）
├── 第一部分：[主题名称]
├── 第二部分：[主题名称]
│   ├── [子主题]
│   └── [子主题]
├── 第 N 部分：[主题名称]
└── 行动项

Start writing immediately after presenting; top-level headings start from H2, no need to wait for user confirmation.

Step 7: Generate summary XML

Prompt input to AI：

[场景: {scene}] [发言人映射: {speaker_map}]

新增转写（{N} 句，{时长}秒）：
{speaker}: {text}
...

[人物背景（来自笔记库，供参考）]:
- 张总（来自《Q4 复盘》）：...

请按 {模板名} 模板提取要点，生成 WPS XML 格式的增量摘要。
只写本段新增内容对应的要点，不要重复已有内容。

Constraints: Strictly based on the transcript, skip empty sections, mark ambiguous information as "To be confirmed", reference the user's style, default to rich text formatting.

Step 8: Write back to notes

Core principle: New summary content is always inserted before the temporary area hr; the temporary area always stays at the bottom.

First round of writing(temporary area does not yet exist):

# 1. 插入摘要内容到笔记末尾
wpsnote-cli edit --json-args "{
  \"note_id\": \"$NOTE_ID\",
  \"op\": \"insert\",
  \"anchor_id\": \"$LAST_BLOCK\",
  \"position\": \"after\",
  \"content\": \"<h2>转写摘要</h2><p>...</p>\"
}"
# 记录 summary_anchor_id = 本次最后插入的 block_id

# 2. 在末尾创建临时区
wpsnote-cli edit --json-args "{
  \"note_id\": \"$NOTE_ID\",
  \"op\": \"insert\",
  \"anchor_id\": \"$SUMMARY_ANCHOR_ID\",
  \"position\": \"after\",
  \"content\": \"<hr/><highlightBlock emoji=\\\"💬\\\" highlightBlockBackgroundColor=\\\"#EBEBEB\\\" highlightBlockBorderColor=\\\"#C5C5C5\\\"><p><span fontColor=\\\"#757575\\\">把你的想法、补充、纠正贴到这里，AI 下轮自动合并到正文对应位置。也可以直接改上面的正文。</span></p></highlightBlock>\"
}"
# 记录 temp_zone_anchor_id = highlightBlock 的 block_id

Subsequent rounds(temporary area already exists):

# 新内容追加到 summary_anchor_id 之后（临时区 hr 之前）
wpsnote-cli edit --json-args "{
  \"note_id\": \"$NOTE_ID\",
  \"op\": \"insert\",
  \"anchor_id\": \"$SUMMARY_ANCHOR_ID\",
  \"position\": \"after\",
  \"content\": \"<p>新增摘要内容...</p>\"
}"
# 更新 summary_anchor_id = 本次最后插入的 block_id

Step 9: Rescan the outline after writing (required every round)

After each round of writing is complete,immediately re-fetch the note outline, and check chapter by chapter whether the content is in the correct position:

OUTLINE=$(wpsnote-cli outline --note_id "$NOTE_ID" --json)

Check items：

  • Whether each h2 section title is consistent with the content map (no stray sections should be added after the first round is established)
  • Whether the content added in this round falls under the corresponding section, rather than all being piled up at the end of the note
  • Whether the temporary area (hr + prompt block) is still at the very end

When anomalies are found：

  • If new content was incorrectly inserted at the end instead of the corresponding section, use batch_edit to move it under the correct h2 section
  • If the section title order is messed up, after fixing it update summary_anchor_id to the correct block_id of the current section's end
  • When outputting the briefing, note "⚠️ This round fixed N position errors"

Step 10: Handle user input in the temporary area

Check whether the user has written content in the temporary area (any block after the prompt block is user input):

temp_anchor_id = state.get("temp_zone_anchor_id")
if not temp_anchor_id:
    pass  # 临时区尚未创建，跳过
else:
    # 读取 temp_zone_anchor_id 后方 10 个 block
    # 过滤掉 hr 块和含「把你的想法」的提示块
    user_blocks = [b for b in after_blocks
                   if b["type"] != "hr"
                   and "把你的想法" not in b.get("content_text", "")]

If there is user input, perform merging:

  1. Feed the user content + current summary outline to AI, and determine where each item should be inserted (supplement/new topic/correction)
  2. Use batch_edit to insert/replace at the corresponding position in the main text
  3. Delete the block written by the user in the temporary area, keep <hr/> and the prompt block

Merge prompt：

用户写在临时区的内容：
{user_temp_content}

当前笔记摘要大纲（含 block_id）：
{summary_outline}

请判断每条内容应插入到哪个 block_id 之后，生成对应 WPS XML。
规则：补充 → 插入对应观点之后；新话题 → 插入摘要末尾（临时区 hr 之前）；纠正 → 替换对应 block。
保持原有排版风格。

Step 11: Detect *** completion requests

STARS=$(wpsnote-cli search --note_id "$NOTE_ID" --query "***" --json)

If ***is found, read the 3 blocks before and after it, generate the completion content, then replace:

wpsnote-cli edit --json-args "{
  \"note_id\": \"$NOTE_ID\",
  \"op\": \"replace\",
  \"block_id\": \"$STAR_BLOCK\",
  \"content\": \"<p>补全后的内容</p>\"
}"

Step 12: Update the status file

if new_sentences:
    state["last_end_time"] = new_sentences[-1]["end_time"]
    state["summary_anchor_id"] = last_inserted_block_id
    if state["temp_zone_anchor_id"] is None and temp_zone_block_id:
        state["temp_zone_anchor_id"] = temp_zone_block_id
state["round"] += 1
json.dump(state, open(state_path, "w"), ensure_ascii=False)

Step 13: Output this round's briefing

✓ 第 3 轮完成（新增 18 句 / 142 秒 → 摘要 +85 字）
  场景：会议记录  发言人：张总、李工❓
  人名联动：张总→《Q4 复盘》、李工→未找到
  行动项：+2 条  ***补全：1 处  临时区合并：1 条
  下一轮：60 秒后

Scenario identification table

Scenario keywords Scenario type Template used
Meeting, discussion, weekly meeting, review Meeting record Meeting minutes template
Class, lecture, teaching, teacher Class notes Class notes template
Training, instructor, trainee Training course Training course template
Knowledge sharing, sharing session, experience sharing Knowledge sharing Knowledge sharing template
Livestream, streamer, audience Livestream content Livestream record template
Podcast, guest, program Podcast/video Podcast Minutes Template
Interview, journalist, interviewee Interview Record Interview Record Template
Negotiation, Party A, Party B, contract Business Negotiation Business Negotiation Template
Retrospective, summary, milestone Project Retrospective Project Retrospective Template
Court hearing, plaintiff, defendant, courtroom Court Hearing Record Court Hearing Minutes Template
Patient, diagnosis, medical orders, symptoms Case Record Case Record Template
Dictation, article, blog Dictation to Article Article Template
Phone, call, customer service Phone Recording Phone Record Template
No obvious characteristics General Scenario General Transcription Template

Formatting Guidelines

Rich text formatting is used by default. Only fall back to plain paragraphs when the user explicitly says "plain text" or "no formatting".

Color Block Semantics

columnBackgroundColor：

  • Blue #EBF2FF: neutral information, speaker's viewpoint
  • Green #E8FCEF: conclusions, consensus
  • Yellow #FFF5EB: notes, to be confirmed
  • Red #FFECEB: risks, issues, points of dispute
  • Purple #FAF0FF: supplementary information, extended content
<columns>
  <column columnBackgroundColor="#EBF2FF">
    <h4>发言人 · 身份信息</h4>
    <p listType="bullet" listLevel="0">要点一</p>
    <blockquote>「原话片段」</blockquote>
  </column>
</columns>

Usage Judgment

  • Use color blocks: there are clear speakers and 2+ viewpoints; a topic has more than 3 key points
  • Do not use color blocks: only 1 piece of information; action items/Todo; when content is minimal and the structure is simple
  • Two columns: only used for scenarios with naturally opposing content (both parties in a negotiation, comparison of pros and cons)

Structural Principles

First divide into h2/h3 sections by topic/issue, then use color blocks within sections to distinguish speakers:

h2 主题一
  ├── columns(蓝) 发言人 A
  ├── columns(蓝) 发言人 B
  └── columns(绿) 结论/共识
h2 行动项
  └── todo list（不套色块）

Scenario Templates

All templates follow the principles of "strictly based on transcription, skip empty sections, rich text formatting by default".

Scenario Main Structure Special Requirements
General h2 summary → h3 topic → columns(blue) → todo —
Meeting Minutes h2 minutes → h3 agenda item → columns(blue) speakers → columns(green) conclusions → todo Includes attendees, decision annotations
Class Notes h2 notes → h3 knowledge points → h4 details → h3 key points → todo homework No color blocks, use hierarchy instead
Knowledge Sharing h2 record → h3 core knowledge → h4 knowledge points → h3 key points → todo learning —
Livestream/Podcast h2 minutes → h3 topic → columns(yellow) Guest A → columns(blue) Guest B Indicate guest identity
Business Negotiation h2 record → h3 agenda item → two columns(blue/red) Party A and Party B → columns(green) agreement → todo Two-column opposing layout
Project Retrospective h2 Review → h3 Results → h3 Issues → h3 Lessons → todo Improvements Quantified results
Court hearing/case/interview/phone call/training Organize according to the sequence of each professional workflow, see the explanation below —

Professional scenario structure：

  • Court hearing: Pretrial preparation → Court investigation → Evidence presentation and cross-examination → Court debate → Final statement
  • Case: Chief complaint → History of present illness → Diagnosis → Treatment plan → Medical orders
  • Interview: 5W1H elements → Notable quotes → Follow-up
  • Phone recording: Purpose of call → Discussion content → Commitments → Follow-up actions
  • Training course: Course outline → Knowledge points → Practical exercises → Homework
  • Dictation to article: Introduction → Chapters → Conclusion (spoken to written language)

See the complete XML template at templates.md

Todo extraction rules

Identify tasks from new transcriptions:

  1. Explicit tasks: Contains action verbs (complete, submit, prepare, follow up) + time limit or responsible person
  2. Commitments: "I will/we will..." "I'll give you... at XX time"
  3. Learning tasks: "Need to learn/practice/apply..."
<p listType="todo" listLevel="0" checked="0">[任务]（负责人：XX，截止：XX）</p>

Stop conditions

  • The user explicitly says "stop", "end summary", or "stop"
  • For 5 consecutive roundsno new transcriptions
  • Audio status=stopped and the final summary has been completed

Clean up state files when stopping:rm /tmp/lts_state_{note_id}.json

Error handling

Error Handling method
CLI unavailable Fall back to MCP mode
NO_ACTIVE_EDITOR_WINDOW Prompt the user to open the note, wait 30 seconds, then retry
WEBSOCKET_NOT_CONNECTED Wait 10 seconds, then retry transcription retrieval
EDITOR_NOT_READY Wait 2 seconds, then retry writing
BLOCK_NOT_FOUND Use outline to refresh the ID, then retry; check whether content is a string
DOCUMENT_READ_ONLY Inform the user that the note is read-only, stop writing but continue reading
Transcription returns empty Wait 30 seconds (it may be transcribing) and retry
Writing repeatedly fails Check whether content is an XML string, and cannot be an array/object; refresh block_id and retry