speaker-labeling

Speaker labeling

What the user wants: a real name before every sentence in the transcript, subtitles that show who is speaking, and exported transcripts and voiceovers split by person. This takes two steps: first work out how many voices there are and which sentence belongs to whom (separation), then give each voice a real name (labeling). Separation relies on a speech model, labeling relies on evidence; neither step relies on guessing.

The easiest way to get this wrong is guessing: seeing two guests listed in the description and filling in "Speaker 1" and "Speaker 2" in order. Wrong names then appear in every subtitle and every exported file, and users often only notice much later—far worse than leaving "Speaker 1" in place.

1. Check the current state first

Use videos_inspect to find this video's transcript (the document whose kind is speech; if there are several, match the source material to the segment the user mentioned), then read it with documents_read:

The body's speakers[] lists each speaker's id and name, and each word carries its speaker.
Already separated(two or more speakers): skip to section 3 and only do labeling.
Not separated(only one speaker, or words carry no speaker), but the user says more than one person is in the video: do section 2.
If the user only wants to change names and does not want to re-run, also only do section 3.

2. Separation: re-transcribe using voiceprints

Separation requires re-transcribing, and it replaces the current transcript rather than creating a separate one. This step discards any text the user manually edited in the transcript, so explain this clearly before proceeding:

First tell the user: re-transcribing will replace the current transcript, and any text manually edited in it afterward will be lost; translations, subtitles, and voiceovers will carry over based on the new transcript. Continue only after the user agrees.
Use transcribe for this video, withdiarize set to true andtarget set to replace; reuse this transcript's language and hints (names, terminology), or use the new ones if the user provided them.
If the model cannot separate speakers, the task still completes, with a diarization-unavailable warning in the result: tell the user this model cannot separate speakers and that they need to install a "speaker separation" model package or switch to a model that can; do not pretend separation succeeded.
If the transcript was modified by the user after transcription, the replacement is rejected with TRANSCRIPT_EDITED : tell the user this, and only add acceptEdited and retry once the user confirms discarding those modifications.
After the task completes, run documents_read again to see how many speakers were separated and how much each one said.

3. Labeling: give each speaker a name

Before acting, gather the evidence for each speaker, then decide the names together:

Placeholder names: the "Speaker n" auto-assigned by the transcription flow and its equivalents in other interface languages (Speaker 1、Sprecher 1 patterns like "speaker + number"), numbered labels from the recognition service (SPEAKER_00、S1、spk-1), and names that are empty or equal to the id. Other names were chosen by the user; leave them alone.
Where evidence comes from: the opening passage of the transcript; the few sentences around each speaker's first utterance; the title, publisher, and description of the source material (videos_inspect's assets[].sourceOn-screen name tags: use videos_frames to look at the first one or two minutes and a few frames at each person's first speaking moment; the name tag must clearly indicate that this person is speaking—a name that merely appears somewhere on screen does not count.
What counts as evidence: self-introduction ("I am…", "my name is …"); introduction by another party ("today we have …", "welcome, …"); a response by this person immediately following being named; a signature-style sign-off; source information that uniquely identifies one of them. When the metadata lists several guests but it is impossible to tell which voice is whose, evidence from the transcript is also required. Speaking order, speaker numbers, and the topic itself do not count; conversely, when the title or description names the participants, you must find evidence for each person individually and match names to voices.
How to write names: use the form actually used in the material, usually the full name, or the form that appears when there is no full name; preserve the original text and spelling, do not translate or transliterate; do not add titles, honorifics, organizations, parentheses, quotation marks, or explanations; no more than 40 characters. When there is no name but a role is explicitly stated (host, interviewer, narrator), a short role label may be used; do not invent a role based merely on how the conversation looks.
Do not guess: when unsure, keep the placeholder name.
One person with several ids: be alert to the possibility of splitting one person into several ids, or merging two people into one. When each has evidence, give several ids the same name without merging the ids; when the voices differ greatly and you are unsure whether it is the same person, ask the user first.

Compare (illustrative only):

Bad: the description says "Guests: Zhang San, Li Si", so "Speaker 1" is changed to Zhang San and "Speaker 2" to Li Si.

Good: at 00:12 the host says "today we have Zhang San", and immediately "Speaker 2" answers "hello everyone", so "Speaker 2" is Zhang San; "Speaker 1" keeps asking questions, no one has called his name, and the description does not say who the host is, so keep the placeholder name, or label it "Host" as the user wishes.

4. Confirm first, then write back

Before writing back, show the user the plan: the name chosen for each speaker, the evidence for it (one sentence), and the moment of their first speaking and that sentence, so the user can jump to it in the editor and listen.
After the user confirms or changes the names, use edits_apply's renameSpeakers, giving documentId and speakers as [{id,name}], all in the same transaction; use the video version from videos_inspect for expectedRevision, and reuse the commandId on retries. Do not upload words or the whole body text. A version conflict means the user just made changes: re-read, verify the IDs and names, and redo with a new commandId.
When only names are changed, the translation does not need to be redone, and the speakers in the subtitles follow the transcript.

5. What to avoid

Do not fill in names according to the order of the list in the description or according to who speaks first.
Do not re-transcribe without notice; do not pretend speakers have been distinguished when the model cannot distinguish them.
Do not add titles, organizations, or parentheses to names; do not translate or transliterate names.
Do not touch names the user has chosen; do not merge ids.
Do not change the words of the body text: only speakers are changed here.

6. Delivery

How many speakers were distinguished; if re-transcribed, state which transcript was replaced and what content was carried over.
Who was given a name and on what evidence; which ones have no evidence and remain placeholder names.
Where the speaker distinction is wrong (a different person within one sentence, two people merged into one), list the timestamps and ask the user to manually adjust them in the speaker settings in the editor.