shorts-segments

Cut short videos from a long video

People who come across this short video haven't seen the long video and don't know what was said before. If the first three seconds don't grab them, they swipe away; if "that thing mentioned earlier" comes up in the middle, they won't know what it refers to. So a short video has to stand on its own: the opening is the hook, it covers only one thing, and it stops on the conclusion.

The unit to pick is thetopic: one thing from being raised to being finished. One point and its reasons, one question and its answer, one story and its landing. One short video is one complete stretch of one topic, no more and no less.

The two easiest ways to pick badly:

Cutting by the long video's chapters. A chapter is ten minutes, it takes three sentences of setup before getting to the point, it ends on "next let's look at", and it contains two topics. That's a stretch of the long video, not a short video.
Only looking for punchy lines. A striking sentence, padded with a few seconds before and after to reach the desired length: the topic has just started or hasn't landed yet when it cuts off, and viewers hear a conclusion without knowing why, or only hear half of it.

What's delivered iseditable sub-videos, not mp4s: each stretch is built with videos_derive as a separate video alongside the parent video, and the user keeps adjusting in and out points, visuals, and subtitles in the editor, then exports when they want files.

1. Get the transcript first

If there's no transcript (a document with kind speech), transcribe first and tell the user transcription is in progress. Picking clips only needs word timings from the source text; polishing and translation are done only when the user asks. If the video already has chapters, chapters help locate topics, but chapter boundaries are not short-video boundaries.

2. Read the transcript and find topics

Read the timed transcript: export subtitles (kind subtitles、format srt) or a transcript with timecodes (kind transcript、format md、timestamps true), give purpose preview(just for you to read, not to hand to the user), wait with jobs_inspect until completed, and read the file in outputs. These times are times on the edited timeline, and they use the same clock as videos_derive's start, end, and frame capture.

How to determine sentence boundaries: to find where sentences start and end, you don't need to dump the entire speech document. When documents_read reads the transcript, the returned translationBasis.sentences gives each sentence's start and end (in seconds), but that's thesource clock; videos_derive's start and end are on thetimeline clock. When the parent video plays this source from 0 seconds, at normal speed, with no cuts, the two are the same, and you can use the sentences' start and end directly to define the range. Confirm this first: use videos_inspect (or facts) to check whether the video item's sourceIn is 0, whether the speed is 1, and whether there are any cuts. If any of these isn't satisfied, use the timeline times from the SRT or transcript you exported above, or convert source times to timeline times before passing them in; don't use source times directly as timeline times.

When the user has picked a topic("cut one about X"), first use space_search_v2 (query To write X means,videoIds only for this video,kinds give ["speech"]) find the several places where it is mentioned, then use the transcript before and after those passages as described above to determine the interval; when the hit time is on the current timeline (clock is sequence), it can be used directly as timeline time;needsResolve first use space_resolve_search_hit to relocate according to the current version;weak first read the original text. When there is no semantic index (effectiveMode is lexical), switch to a few other keywords and search again, or read the full text.

Already cut: space_list gives videoId list the short videos derived from this video, read their names and intervals, and do not cut the same segment again; if the user wants to cut another one on the same topic, ask first.

First draw a topic map

Read through the entire transcript (if the user gave a range, read that passage), and split it by topic: when the question, viewpoint, or story changes, that is the start of a new topic. For each topic, note the approximate start and end, a one-sentence description of what it talks about, and where the landing point (conclusion, answer, or punchline) is. A chapter often contains several topics, and a topic may also cross chapter boundaries.

Pick the best topics

A qualified topic must satisfy all three conditions; if one is missing, do not use it:

Has a highlight: at least one of these four: a counterintuitive statement, an emotional high point, dense useful content, or a short story with a twist.
Stands alone: understandable without watching the long video, with no references such as "that thing just mentioned" or "the second point" that require watching earlier parts to understand.
Has a beginning and an end: the topic contains a sentence that can directly open it (see "the opening sentence is the hook" below), and it also reaches the landing point.

The transcript is only a clue: for qualified topics, use videos_frames to capture a few frames at those seconds to look at the visuals (whether there is anything worth watching, whether the speaker is in frame), then combine with the context to judge whether it is said seriously or as a joke. Frames are source footage and do not include subtitles or overlaid elements. Then sort by highlight strength and how solidly it stands, with the strongest first.

How many

The user gave a number(if the parameter page is prefilled, it states the number, for example "want N"): take the strongest N topics. If fewer than N topics qualify, only create the qualified ones, do not pad with weak topics, and tell the user why there are fewer.
Automatic(if the prefilled text lets you decide how many based on topics, for example "you decide how many based on topics", or the user did not say): create as many as there are qualified topics, do not pad the number, and do not lower the threshold just because there are many topics. If there are more than 10, first create the strongest 10, and list the rest in the reply as candidates (in point, out point, one-sentence description of what it talks about), and create them if the user wants.
Two clips do not share the same passage: when two candidates overlap, keep only the stronger one.

How long each clip is

Automatic(if the prefilled text lets you decide the length based on topics, for example "decide the length of each clip based on the topic", or the user did not say): the length is the length of that topic. The start is the first sentence in the topic that can directly open it, and the end is after the landing-point sentence is finished. Do not include setup or the beginning of the next topic just to make the duration, and do not cut out the reasons or examples in the middle just to reduce the duration. If a long off-topic passage is inserted in the middle of the topic and it drags on too long, take the half of the topic that stands alone and also has a landing point, or switch to a tighter topic.
The user gave a range(possibly 15–30, 30–60, 60–90 seconds, 1–3 minutes, or custom): find a topic within the range that can be fully covered; when the topic is longer than the range, take a small point within it that can stand on its own and has a landing point. A topic that does not fit and cannot be split is better left out than cut in half; going a few seconds over the range in order to stop at the end of a sentence is acceptable, state this in reason.
No platform limit: if over 3 minutes, just mention it once; YouTube will not treat it as a Short, everything else proceeds as usual. When the automatically determined length exceeds 3 minutes, first check whether it contains two topics; if it can be split, split it into two clips.

Boundaries

One short video covers only one thing: when a segment contains two topics, split it into two clips, or keep only one.
The opening sentence is the hook: the start of a topic often has preamble ("So let's talk about...", "Actually, this question..."); move the starting point to the first sentence in the topic that can directly grab attention: a counterintuitive conclusion, a specific number, a question. After moving it, the following words must still be understandable; if not, you moved too far, or this topic has no good opening—pick another one.
End on a landing point: stop at the sentence where the conclusion or punchline finishes, not on a half-sentence like "Next let's look at..."; when the ending is something like "follow, subscribe", move the endpoint earlier to the previous sentence.
Someone must be speaking in the first 3 seconds: if not, move the starting point to the first sentence; do not add a title card in front.
In and out points fall on sentence boundaries: the in point is the midpoint of the pause between the first word of the opening sentence and the word before it; the out point is the midpoint of the pause between the last word and the next word. Do not cut into a word. When the boundary is uncertain, loosen it slightly: it is easier for the user to trim in the editor than to add back.
When the pause is shorter than one frame (about 40 milliseconds), "take the midpoint of the pause" does not hold: the midpoint does not land on a whole frame, and rounding will eat into the next word or swallow the previous word. At runtime the boundary is now pulled into the pause (the end point does not eat the start of the next word, the start point does not swallow the end of the previous word); the returned range is the value actually used and may differ from what you passed by a few frames. After creating, check the difference between the returned range and what you wanted: if it differs a lot, or the words at the start or end are cut off because of it, create again with a different boundary; do not leave it as is.
Interviews: prefer a whole uninterrupted stretch of one person. It is allowed to start from the first complete sentence of the answer. Sentences at the start that depend on prior context ("Another way to put it is", "This is like what was just said") should not be used as the opening; move later to the sentence that can stand on its own. It is acceptable for the interviewer to interject one or two short remarks, but state in the reason at which seconds and what was said.
Station logos, borders, and watermarks are a go/no-go check: after extracting frames, check the corners and edges for station logos, sponsor bars, burned-in subtitles, and borders. When the crop to be used would cut them in half or leave half a character, avoid them according to the rule in step 4 framing for avoiding station logos (the crop avoids the station logo entirely, and top and bottom borders are cropped out); if it cannot be avoided, switch clips and tell the user what choice was made; do not deliver with half a station logo left in.
Do not look again for what has already been cut: when the user mentions it (the prefilled text includes "do not look again for what has already been cut") or you suspect it has been cut, use space_list (set videoId to the parent video) to see the derived child videos (origin.source is derived) and their intervals; new candidates should avoid these intervals.
The video has been edited and is out of date, and the times in the transcript are already post-edit times; when the user wants to cut based on the pre-edit original footage, first confirm with the user which version to use.

Compare (just for illustration):

Bad:12:00–14:30, starting from "Okay, so next let's talk about the cost side" and ending at "we'll expand on this later."

Good:12:41–13:22, starting from "We calculated that the cost of one video is actually only three yuan" and ending at "so what's expensive has never been generation, it's rework."

The latter segment is a complete topic: the first sentence is the number, the middle is the reasoning, and it stops on the conclusion, with no references to earlier text.

3. Create first or show the user first

If the user's words carry a meaning like "once you've found them, just create them into videos directly, no need to ask me first" (the parameter page prefill includes this sentence by default): once found, go directly to step 4, and do not confirm segment by segment in the conversation.
If this sentence is absent, or the user says "show me first": first list the candidates and wait for the user to choose. For each segment, write the in and out points, which topic it is about (one sentence), the opening line verbatim, the highlights, why it can stand alone, and any problems found (for example, burned-in subtitles on the picture, split screen or side-by-side two-camera setup, a corner logo or sponsor bar that would be half cut off when cropping to vertical; in an interview, when the other person is speaking, the speaker is not in the picture). If frames were extracted, say which spots were looked at; if the picture was not viewed, say it was not viewed. Sort by confidence from highest to lowest, and create only after the user has chosen; when the user says "you decide," decide yourself and explain the reasoning.

There is no need to repeatedly ask about boundaries in the conversation: the sub-video retains the entire timeline and transcript, and the user can bring back a sentence they crossed out in the editor. Write the title and reasoning in the language the user speaks.

4. Create sub-videos

Call videos_derive for each segment:

video is the parent video,start、end is seconds on the post-edit timeline,name use the segment title.
Aspect ratio: when the user wants 9:16, width 1080、height 1920; for 1:1, 1080×1080; when keeping the original aspect ratio (or if not mentioned), do not pass width, height, or fit. Do not upscale small dimensions for the sake of "high definition."
Framing: if an aspect ratio was passed, give fit: "cover"(fill and crop out the excess). If the user says "centered": do not pass crop and crops, create it centered; if they say "I'll decide myself": likewise create it centered, and tell the user they can drag and zoom the picture directly on the canvas to position it themselves (as soon as they drag, it becomes free placement, and afterward changing the aspect ratio will no longer fill and crop accordingly). If the user did not say or says "automatic" (default): follow the shot-by-shot framing process below. The crop box given is only framing: in the sub-video it is stored as a framing point, the source picture is not cropped, and after changing back to the original aspect ratio the full picture can still be seen. Framing is a static crop box switched by shot, not a motion shot that follows the person: when the user wants "the picture to keep moving with the person speaking," truthfully say it cannot be done. Segments with framing points can only be zoomed in and out around the framing point on the canvas (focalPoint.zoom), and cannot be dragged: to change the position a certain segment is aimed at, use setStyle to change that segment's focalPoint, or ask the user to change it to free placement on the canvas and position it themselves (afterward, changing the aspect ratio will no longer automatically aim).
How to do framing "automatic"(the prompt says: see where the speaker is in the picture and put the crop box on them; if no person is found, center it): first split by shot, and for each shot determine the crop box based on the face detection result, without relying on looking at the picture to estimate the position.
1. First split into shots, then extract frames: first use videos_shots's range（起:止, take this segment's shots(each shot has start、end and three sampleAt) and cuts. It only detects picture changes, it does not recognize people. Then use videos_frames's at to pass all shots' sampleAt, 8 shots at a time (24 frames, which is the single-call limit). Each frame's faces is the detected face box,x、y、w、h normalized to 0–1 by the frame's width and height, corresponding to the source picture, exactly crop 's coordinates.
2. First split by camera position: for each frame look at the area of the largest face (w × h). Those not less than 0.01 areclose-up frames; those with only smaller faces arewide frames(long shots with several people in frame); those with no face are recorded separately. The three frames in a shot are generally the same camera position.
3. Identify the main speaker: combine all shots in the whole segment, look only at close-up frames, group faces into people by center horizontal coordinate (x + w/2), those differing by less than 0.1 count as the same person. The one appearing in the most frames is the main speaker; when frame counts are close, take the one with the larger average area. The other person's close-up is the interviewer's (or the other party's) shot.
4. First look at the edges, then decide the crop box: open a few frame images, see where the station logo, watermark, subtitle bar, and sponsor bar are, and estimate their edges (0–1, the same coordinate system as crop ); then see whether there are borders burned into the picture at the top and bottom (color bars, black bars).
5. Decide the crop box shot by shot: use this shot's own three frames, do not substitute another shot's position. The cropped width ratio cw = (画布宽 ÷ 画布高) × (原画高 × (1 − top − bottom) ÷ 原画宽)；left = 中心 − cw/2, limited to between 0 and 1 − cw ,right = 1 − cw − left. Example: a 16:9 original producing 9:16 with no top/bottom cropping,cw = 0.316; when the face center is at 0.30, give left 0.142, right 0.542. When there are borders burned in at the top and bottom,top、bottom give the border's height ratio to crop it out (give a bit more, do not leave a thin edge), if none give 0.
  - Main speaker close-up: take the center as the median of the main speaker's face center horizontal coordinate (x + w/2) in this shot.
  - Interviewer close-up: take the center as the interviewer's face, so that the person currently speaking is in the picture; this is exactly the point of framing by shot.
  - Wide shot (several small faces, the picture is mostly the room): use the smallest crop box that can contain everyone. When all faces' outer edges (the leftmost x to the rightmost x + w) can fit into a window of width cw , take the center as the midpoint of this group of people; when they cannot fit, state it in notes, and take the center as the main speaker.
  - When faces are very small in a long shot, crop at most 15% from the top and bottom (top、bottom both not exceeding 0.15) to make the faces larger, not exceeding that; when there are burned-in borders, the border height also counts within this 15%, and if the border is taller, the border takes precedence, and explain it in notes.
  - When adjacent shots' crop boxes differ by less than 0.02 (the difference inleft , and top、bottom the same), merge them into crops one item,from take the previous shot's starting point,to Take the endpoint of the latter shot.
6. Avoid the station logo: Check shot by shot; the threshold is the same as before. When the station logo is on the left,left must be at least the logo's right edge + 0.01; when it is on the right,left + cw must be at most the logo's left edge − 0.01 (right must be at least 1 − the logo's left edge + 0.01). If after moving it the face in this shot is cropped out of frame, or neither side can avoid it, then changing the crop for this shot will not work either, so change the segment; do not deliver a crop that presses against half the station logo.
7. Multi-camera: Wide shots and interviewer shots now have corresponding crops, so segments are no longer changed just because of them. Only two rules remain: segmentsmust not start with the other party's reaction shot; when wide shots or empty shots together exceed about 40% of this segment, prefer changing to another segment and explain why; if it cannot be changed, do it as is and state it in the notes.
8. Cut points come from videos_shots：crops 's from、to directly use shots 's start、end, which is frame-accurate; no longer probe 1 second before and after the estimated cut point. Only when you see that three frames of a shot are clearly two camera positions (meaning a cut was missed), lower threshold to 0.2 and rerun videos_shots for that small section; or use at to fetch a few extra frames around the suspected time point. When creating:crops pass all merged items ({from, to, left, top, right, bottom}, at most 64 items, non-overlapping, all in seconds on the parent video timeline), and also pass the first shot's crop as an ordinary crop as a fallback for frames not covered.
9. No face: faces is empty for all frames; center it (do not pass crop and crops), and truthfully say that no face was found. When the result's detector is null, the detector is unavailable, not that there is no person: in this case fall back to looking at frame images, estimate shot by shot which side of the frame the main speaker is on, give crops and cropaccording to the algorithm in steps 5 and 6, and explain that this time it was estimated by looking at images.
hook、reason、notes: Pass it for every one.hook is the first sentence of the retained segment, copied verbatim, not rewritten (at most 300 characters);reason says in one sentence why it was chosen, written for the user to read (at most 500 characters);notes is an array of notice strings, writing things discovered in framing, camera position, and station logo checks that the user needs to know; if there are none, do not pass it.
- notes Each at most 80 characters, at most 8 items; exceeding this will cause the entire creation to be rejected. One matter per item, write them separately, do not cram several matters into one item.
- When mentioning time, write it as "sub-video second X (original video second Y)", giving both clocks. One item per shot group, writing only what the user needs to know, for example "sub-video seconds 9–24 (original video 24:54–25:09) is a wide shot, cropped to between the two people" "sub-video seconds 40–43 (original video 25:25–25:28) is the interviewer, cropped to the interviewer" "the purple bars at top and bottom have been cropped out". When the beginning is the other party's reaction shot, also write it, as "sub-video seconds 0–3 (original video 24:45–24:48) is the other party's reaction shot".
- The card on the results page will display these three items, so there is no need to repeat them in the reply. After creation, check whether what was written in is what you want: the returned result echoes hook、reason、notes It is the one from the source document it was derived from (when an existing sub-video is recognized, it is the original one; the one uploaded this time will not overwrite it); you can also use documents_read to read the returned derivationDocumentId。
chapters: false: a 30-second sub-video should not carry the parent video's entire chapter list.
captions Follow the user's subtitle requirements: if subtitles are wanted, provide true(default); if not wanted, provide false. When the user wants only the original text or only the translation, choose the subtitle layer according to what they say, and if it cannot be done (for example, there is no translation), say so honestly.
Translation: videos_derive by default carries the parent video's translation and translation subtitle layer over to the sub-video; translations in the old format are automatically upgraded before being carried over. When a translations-skipped warning appears in the return, tell the user which language was not carried over.
Check the return values：range is the boundary actually used; compare how much it differs from what you wanted (see the short pause above);captions[].cueCount counts only the cues that fall within the interval, not the number of entries in the entire subtitle; use it to judge whether the subtitles were created and whether the count is reasonable.
After each one is created, a video card automatically appears in the session; there is no need to paste the path again. Calling repeatedly with the same interval and settings will recognize the existing sub-video (reused is true) and will not create extra ones.

What a sub-video is: it copies the parent video's entire timeline and transcript, the parts outside the interval are cut out, and the user can bring them back by clicking the crossed-out sentences in the editor; the original video is not changed. So do not move any of them to the recycle bin, and do not rebuild one just to "change the boundary"; let the user drag it in the editor.

Removing verbal tics and tightening pauses will change the entire video, so do not do it here; when the user asks for it, first explain the impact, and handle it on the sub-video according to the method for cutting spoken-word content.

Warnings that must be relayed faithfully

Tell the user each warning in the creation result one by one, in plain language:

multiple-assets: the parent video has more than one main asset; this version only cuts according to the main asset, and other assets move along with it; ask the user to check in the editor.
items-not-restorable: other track segments that fall entirely within the cut region will not come back with "restore".
range-not-exact: part of the area outside the interval could not be cut out (the main asset does not cover it), so the in/out points will be slightly wider than requested.
captions-skipped: the subtitle layer was not created; explain the reason and tell the user they can create it again in the subtitle panel.
crops-adjusted: a framing range was shortened or lost (it exceeded the sub-video range, or no frame remained after cropping); the cropped frames are framed according to the adjacent shot or a fallback crop box; tell the user which seconds: they can select that segment on the canvas and zoom in, change it to free placement first if they want to move it, or ask you to change that segment's focalPoint。
animation-split: the picture was split into several segments by framing, and looping animations or volume envelopes will restart at each segment boundary; tell the user at which seconds, and if there is an impact, ask them to check in the editor.
translations-skipped: this translation of the parent video could not be carried into the sub-video (the old format cannot be converted), so the sub-video does not have this language; tell the user which language it is, and they can translate again on the sub-video.

For rejection errors (the parent video has code frames, the interval is empty, etc.), state them honestly and do not work around them.

5. Finished file

The default delivery is the sub-video, not an mp4. When the user explicitly asks for a file, call export on the sub-video: kind video、format mp4, wait with jobs_inspect until completed, and tell the user the path in outputs. Subtitles on the timeline are burned into the finished video by default (burnCaptions); when subtitles are not wanted, provide falseIf the user wants to export directly without creating sub-videos, they can also use the old method of exporting by ranges on the parent video, one per segment {start, end}。

6. What to avoid

Don't cut by chapter, and don't let one segment contain two topics.
Don't pad the duration around a single punchy line and cut off before the topic is finished; don't lower the bar or include setup just to reach a target number of clips or duration.
Don't start from the setup, and don't stop mid-sentence or on "follow and subscribe."
Don't leave in references like "what I just said" or "the second point."
Don't cut into the middle of a word; don't add a title card at the front.
If the user didn't say "show me first," don't stop segment by segment to ask about boundaries; if they did say it, don't decide for the user unless they say "you decide."
Don't say footage you haven't watched is fine; don't swallow warnings.
Don't promise that the framing will always follow the person as they move; framing each shot separately is what's achievable, and the user can select any shot's segment on the canvas to enlarge it, but to move its position they must first switch to free placement; don't use a single crop frame for the whole segment; don't use another shot's face position to set this shot's crop frame; don't estimate the seconds of a cut from a few frames, use what videos_shots returns.
Don't pass the sentence time from the source clock as timeline time directly; don't leave a returned range that differs greatly from what's wanted without checking it.
Don't let a clip start on the other person's reaction shot or on a sentence that depends on prior context; don't deliver a crop frame that cuts off half a station logo or keeps a burned-in border.
Don't cram several things into one note, and don't write only one clock's seconds.
Don't export an mp4 and call it delivered unless the file is exactly what the user wants; don't move sub-videos to the trash.
Don't leave inspection artifacts in the user's files: use videos_frames to view footage, not ffmpeg to take screenshots into exports/; anything exported to read the transcript, preview audio, or check the result should have purpose preview, and don't export clips as mp4 by ranges just to check them.

7. Delivery

At the end, list each clip for the user:

Title, in-point and out-point in the parent video, duration, aspect ratio, and resolution.
How many clips or how long each is, you decide based on topic: say one sentence on why it's these clips (for example, the other two topics only make sense with prior context and have no landing point); when there are more than 10 qualifying topics, list the candidates that weren't created.
The opening line and why it was chosen have already been passed to the result card via hook / reason, so don't repeat them clip by clip in the reply; only write one sentence per clip when the user is viewing somewhere without a results page (web or command line).
What was done and what wasn't: one sentence each for aspect ratio and framing (original aspect ratio / automatic per-shot framing / user-defined; any shot's segment can be enlarged on the canvas, but to move its position they must first switch to free placement), and subtitles (created / not created).
Warnings at creation time, or write none if there are none.
Footage issues found (burned-in subtitles, station logos or sponsor bars, split screen or adjacent frames, speaker not in frame when interjecting): write them if there are any; if you watched and found none, write "none seen"; if you didn't watch the footage, write that you didn't watch.
Final sentence: sub-videos can still be adjusted in the editor, and to put back a crossed-out line just click that line; ask again if you want an mp4.