xiaoyuzhou-podcast-transcriber

Xiaoyuzhou Podcast Transcriber

This Skill automates the process of converting Xiaoyuzhou podcast episodes into high-quality text archives.

Capabilities

1. Download: Extract and download audio from a Xiaoyuzhou episode page.
2. Transcribe: Use a locally deployed Alibaba FunASR (Paraformer) model to achieve high-accuracy Chinese speech recognition.
3. Process: Generate three levels of documents:
   • transcript_raw.txt: Word-for-word raw text (Raw Text).
   • podcast_full_structured.md: A structured transcript that preserves the full text but improves readability (including headings and paragraphs).
   • podcast_optimized.md: A condensed, polished summary (Summary) and Show Note.

Prerequisites

• Python 3.8+
• FFmpeg (must be installed and added to the system PATH)
• Internet Connection (used to download audio and model files)

Workflow Instructions

Please perform the tasks in the following order.

Phase 1: Preparation & Download

1. Extract Audio URL：

   • Obtain the HTML of the provided Xiaoyuzhou episode page.
   • Find the audio file URL (usually in the mp3, m4a format). The characteristic pattern usually contains https://media.xyzcdn.net/...。
   • Tip: Use curl to fetch the HTML and use grep to extract the URL.

2. Download Audio：

   • Download the audio file as podcast.m4a (or podcast.mp3)。

3. Format Conversion & Chunking：

   • Convert the audio to 16kHz mono WAV format (required by the FunASR model).
   • Split the audio into 5-minute chunks to ensure stability and reduce memory usage.
   • Example command:

     ffmpeg -i podcast.m4a -ar 16000 -ac 1 podcast.wav
     mkdir -p chunks
     ffmpeg -i podcast.wav -f segment -segment_time 300 -c copy chunks/part%03d.wav
     

Phase 2: Local Transcription with FunASR

1. Environment Setup：

   • If it does not exist, create a Python virtual environment (venv)。
   • Install the necessary packages:modelscope, funasr, torch, torchaudio。
   • Note: Use pip install modelscope funasr torch torchaudio。

2. Write the Transcription Script：

   • Create a Python script named transcribe_chunked.py containing the following logic:

     import os
     import glob
     import traceback
     from funasr.auto.auto_model import AutoModel
     
     # Model Definitions (模型定义)
     model_id = "iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch"
     vad_model_id = "iic/speech_fsmn_vad_zh-cn-16k-common-pytorch"
     punc_model_id = "iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch"
     
     try:
         print("Loading models...")
         model = AutoModel(
             model=model_id,
             vad_model=vad_model_id,
             punc_model=punc_model_id,
             disable_update=True
         )
     
         files = sorted(glob.glob("chunks/part*.wav"))
         print(f"Found {len(files)} chunks.")
     
         # Initialize output file (初始化输出文件)
         output_file = "transcript_raw.txt"
         with open(output_file, "w", encoding="utf-8") as f:
             f.write("")
     
         for i, file_path in enumerate(files):
             print(f"Processing {file_path} ({i+1}/{len(files)})...")
             try:
                 res = model.generate(
                     input=file_path,
                     batch_size_s=300,
                     # Add hotwords if relevant to the specific podcast context
                 )
     
                 text_part = ""
                 if isinstance(res, list):
                     for item in res:
                         if 'text' in item:
                             text_part += item['text']
                 elif isinstance(res, dict) and 'text' in res:
                     text_part = res['text']
     
                 with open(output_file, "a", encoding="utf-8") as f:
                     f.write(text_part + "\n")
                     f.flush()
     
             except Exception as inner_e:
                 print(f"Error processing {file_path}: {inner_e}")
                 traceback.print_exc()
     
         print("All chunks processed.")
     
     except Exception as e:
         print("Fatal error:")
         traceback.print_exc()
     

3. Execute Transcription：

   • Run the script in the virtual environment.
   • Ensure that transcript_raw.txt。

Phase 3: Post-Processing

1. Generate podcast_full_structured.md：

   • Goal: Create a reading-friendly full text (Full Text) version.
   • Input：transcript_raw.txt。
   • Instructions：
     • Read the raw transcript text (Raw Transcript).
     • Do not delete content. Preserve the original wording (including spoken filler words that add naturalness, but you may slightly adjust stuttering and repeated parts).
     • Add Structure: Split long passages into logically clear paragraphs (Paragraphs).
     • Add Headings: Insert descriptive H2/H3 headings for different topics/sections.
     • Formatting: Use Bold to emphasize key points.

2. Generate podcast_optimized.md：

   • Objective: Create a high-quality "Show Note" or Essence Summary.
   • Input：transcript_raw.txt or podcast_full_structured.md。
   • Instructions：
     • Remove Noise: Remove spoken filler words (such as uh, um, you know), repeated sentences, and irrelevant small talk.
     • Refine: Polish sentences into fluent, professional Written Chinese.
     • Summarize: Organize into clear sections with headings.
     • Highlight: Extract key insights or "Golden Sentences".

Phase 4: Final Output

Report the successful creation of the following four Artifacts:

1. podcast_optimized.md
2. transcript_raw.txt
3. podcast.m4a / podcast.wav
4. podcast_full_structured.md