SuperAPI AI Directory
Partner中文AI newsFavorites
Skillssubtitle-workflow

subtitle-workflow

Office & productivity

View repository

Updated 10-20 08:10

About this skill

Transcription and Subtitles

The user drops in a link or a file and says "add subtitles." What they want as the final product is an editable video: the media is on the timeline, it has been transcribed and polished, there is a translation when needed, and the subtitle layer sits on the video; afterward they can change a word, move a subtitle, or switch to another language in the editor, and when they want a file, export it from the video. Do not just hand over a subtitle file that was downloaded and then edited: that way they cannot edit it, and it cannot be put back onto the video.

The easiest way to get it wrong: the steps are out of order. Building the subtitle layer right after transcribing, then polishing, means the subtitle layer has to be deleted and rebuilt; translating before polishing makes all the translations outdated; starting over from scratch when continuing across turns makes the user wait twice. So follow this order throughout: confirm → create video and import → transcribe → polish (including real names of speakers) → chapters (side path) → translate → build subtitle layer → export → delivery notes. At each step, wait for the tool's receipt or task completion before going to the next step; when continuing across turns, first use videos_inspect to see which step has been reached, continue from there, and do not repeat completed steps.

1. Confirm First

Before starting, clarify these points; if anything is unclear, ask once rather than guessing along the way:

  • Deliverable: original subtitles, translated subtitles, or bilingual subtitles; whether files are wanted (subtitle files, finished video with subtitles). When the user says "translate into a certain language," they want translated subtitles; when they say "bilingual," they want bilingual subtitles; when they only say transcribe or add subtitles, they want original subtitles.
  • Target language: use a BCP 47 tag (e.g., zh-Hans、en、ja、es). If translation is wanted and the user has not said what language to translate into, ask once.
  • Source language, names, and terminology: note them down if the user mentions them. Give the language to transcription; give names and terminology as hints to transcription when the transcription model accepts recognition hints (see step 3), and when it does not, use them only during polishing. If no language is given, let transcription detect it automatically.
  • Source metadata: for media imported from a link, the assets[].source in videos_inspect contains the title, publisher, description (description, up to 1200 characters), and the author's chapters (chapters); when transcription or download finishes, pipeline.summary.description and pipeline.summary.sourceChapters also give the description and the number of author chapters. Use it for polishing, speaker real names, and chapters.
  • Which video to work on: work on the video open in the editor or named by the user; use videos_inspect to see existing media, transcriptions, translations, and subtitle layers, and reuse what already exists.
  • When the user explicitly wants only a subtitle file and no video or project, do not follow this workflow; follow the files section of translate-subtitles (when only transcribing without translating, deliver the transcript in the download directory).

2. Create Video and Import

  • Link: transcribe give url(can give language, hint, name, noCaptions); it creates a new video, downloads and imports it, puts it on the timeline, transcribes it, and builds the subtitle layer; when the user has stated the source language, include language; when names and terminology are given and the transcription model accepts hints, include hint (see step 3). download give newVideo: true and transcribe: true Also create a new video and transcribe it, but use only the default settings and do not create a subtitle track. jobs_inspect waits until completed:pipeline.summary.videoId is the new video,pipeline.summary.assetId is the asset. If a later step fails, keep the newly created video and do not create it again.
  • Local file: transcribe gives file(you may provide language, hint, name); it creates a new video, imports it, places it on the timeline and transcribes it; the transcription in step 3 happens in this step. When you only want to create a video without transcribing, videos_create creates a new video (name taken from the file name; for portrait assets provide portrait width and height), use assets_import to import it and place it on the timeline (provide place).
  • If link download fails, tell the user how to remedy it according to error.details.remedy, and do not download in any other way.

3. Transcription

  • In step 2, when using transcribe or download with transcribe: true it is already transcribing; when an existing video needs transcription, or when language or hint is needed, use transcribe with video (you may provide asset; first check models_capabilities to see whether a usable transcription service is available).
  • When polishing is needed, add to transcribe noCaptions: true: the subtitle track is created by step 7 after polishing (transcribe by default creates one after transcription; any created before polishing must be deleted and recreated).
  • hint depends on the model: before including hint, first check in models_capabilities the transcription model you will use (the default one when model is not provided) for its acceptsHint. The default MOSS does not accept hints: if provided, it still transcribes as usual, but the hint is ignored, and the transcription task carries hint-ignored as a reminder. In this case do not include hint; leave names and terms for polishing in step 4, and do not switch to a hint-accepting model just to use hint.
  • Transcribe once and use it: do not switch models or re-transcribe just because you feel the recognition is not good enough (transcribe's target: replace). A few typos and misheard names and terms are normal; polishing in step 4 will fix them. Re-transcribe only when the user asks to switch models or re-transcribe, or when the result is unusable (empty, wrong language, large sections of garbled text). When the result is unusable, do not ask first; re-transcribe once directly, and if the language was misidentified include the correct language (automatic detection without it will most likely misidentify again); then tell the user where the first result was unusable, what the re-transcription changed, and how the re-transcribed result is. If it is still unusable, stop and ask the user. One re-transcription takes a few minutes, and the user is waiting.
  • jobs_inspect waits until completed:pipeline.summary.documentId is the new transcription (when transcribe is given url or download with transcribe: true , it is likewise in pipeline.summary.documentId）。
  • When CAPABILITY_NOT_CONFIGURED is returned, tell the user according to its next which service needs to be enabled, and stop here.

4. Polishing

Polish the transcription according to polish-transcript , writing back to the same transcription. By default only fix typos, terminology and punctuation, word for word, without changing word timings; skip when the user says not to polish. Polishing must come before translation: it changes sentence segmentation.

Follow its approach, using source metadata as background reference (accurate spellings of names and terms are only corrected when the speaker is clearly saying them; do not add anything to the transcript); after writing back the polished version, give placeholder-named speakers their real names (only write when there is evidence; if uncertain, keep the placeholder name). When the user says not to polish, still do the speaker real-naming.

5. Chapters (bypass)

By default, chapter the video on every transcription, unless the user says not to, or videos_inspect shows the video already has chapters (do not overwrite; when the user wants re-chaptering, follow video-chapters First ask clearly whether to replace or adjust). Chapters are a side path: they don't block translation, the subtitle layer, or export; failure is not fatal—retry at most once, and if it still fails, honestly write the reason in the delivery notes and continue with the later steps.

  • Author chapters exist（assets[].source.chapters are not empty, or pipeline.summary.sourceChapters is greater than 0, or the description contains 0:00 开场 a timestamped outline like this): use chapters_adopt on the video to absorb the author's chapters onto paragraph or sentence start points in the transcript and write them into the video; don't split them yourself;chapters is empty but a timestamped outline is visible in the description, pass that outline to outline. Read the receipt's sourceChapters: how many are matched, ambiguous, and unanchored; explain at delivery. Chapters with untitled only have boundaries and placeholder titles (Untitled Chapter N); after titling them based on the transcription, call again with outline ; don't write placeholder names.
  • No author chapters(chapters_adopt returns NO_SOURCE_CHAPTERS also means this case and is not a failure): split them yourself following the approach in video-chapters .
  • When the user wants chapters you write yourself, or wants a different granularity from the author's outline, also split them yourself following video-chapters , using the author's outline as reference.
  • When it returns ASSET_NOT_PLACED , the material is not yet on the timeline: add it to the timeline first, then proceed.

6. Translation

When a translation or bilingual output is needed, translate following the approach in translate-subtitles . After polish is written back, re-read the transcription with documents_read and translate using the translationBasis at that time; if a translation in the same language already exists, update it rather than creating a second one.

7. Build the subtitle layer

Use captions_create to place subtitles on the picture; BaoCut aligns them to sentences by time and splits them into subtitle strips by width:

  • Source-language subtitles: give the transcription document as documentId.
  • Translated subtitles: give the translation document as documentId.
  • Bilingual subtitles: give the translation document and include bilingual: true。

Each call on the same document creates an additional layer; it does not overwrite. Before calling, use videos_inspect to check whether this document already has a subtitle layer: if it does, don't create another; if the user wants it redone, first explain that an extra layer will be created and ask whether to delete the old one. The receipt gives a transactionId and the number of subtitle strips; tell the user.

8. Export

When the user wants a file, use export, wait for jobs_inspect to reach completed, and tell the user the path in outputs:

  • Bilingual subtitle file: kind subtitles，bilingual: true(the primary document is the transcription, the secondary document is the translation, one entry per sentence).
  • A subtitle file with only the translation: kind subtitles, provide language; this requires that a subtitle layer in that language already exists on the picture (step 7).
  • Source-language subtitle file: kind subtitles, the primary document is the transcription.
  • Finished video with subtitles: export mp4 or webm; the subtitle layer on the picture will be rendered into it.

9. Things to avoid

  • Don't deliver only a subtitle file unless the user explicitly wants only the file.
  • Don't build the subtitle layer before polishing; don't translate before polishing.
  • Don't re-transcribe just because you think the recognition isn't good enough; don't switch models just to use the hint.
  • Don't repeatedly create subtitle layers on the same document; don't rebuild a video that has already been created.
  • Across turns, don't start over: first use videos_inspect to see how far it got.
  • If chapters fail, don't retry repeatedly, and don't let it block later steps.

10. Delivery notes

Written for the user, making clear what they got and where they need to look for themselves:

  • Where the video is: its name and location; whether it is newly created or the user's original.
  • Subtitle layers: which layers were created (original text, translation, bilingual), and how many entries each has.
  • File: the export path.
  • What the polish changed and which terms were standardized; the target language of the translation and the number of sentences.
  • Speakers: who was given a real name; which ones have no evidence and are still placeholder names.
  • Chapters: how many chapters; whether the author's chapters were used or they were divided independently; entries in the author's chapters that did not match (ambiguous, unanchored); if not done or failed, explain why.
  • Where things are uncertain: transcription errors that were not fixed, translations that are uncertain, and passages with poor recognition quality.