Recently, those AI face-swap videos are easy to come across. Although the characters have changed, the movements feel familiar, and the final video is easy to watch all the way through.
But if you hand the reference video directly to the model, the original actor's facial features, clothes, body shape, and background often get mixed into the final video along with the movements.
LibTV's newly launched deep motion capture solves this problem perfectly.
We upload the reference video,LibTV and with one click it extracts the deep dynamic information from the video, turning the movements, blocking, camera movement, and spatial relationships into a cleaner generation reference, minimizing interference from the original people, clothing, scenes, and art style on the result.
01. What is deep motion capture?
You can think of deep motion capture as a moving spatial sketch.
In an ordinary reference video, the actor's appearance, clothes, background, movements, and camera are all mixed together. A depth video reprocesses the image so that different shades of gray represent the distance between objects and the camera. Who is standing in front, which way an arm is swinging, how far apart two people are, and how the camera pushes in are all preserved; facial features, colors, and background textures are downplayed.
Hand this dynamic sketch to a video model, and a new character can follow the movements and camera work of the original footage.
This type of method is suitable for martial arts, dance, multi-person blocking, and videos that require complex camera movement. The depth information mainly governs how things move, the character reference images determine what they look like, and the prompt then supplements the clothing, scene, and weapons.
02. Deep motion capture comparison test
This test mainly examines whether the deep motion reference can reduce interference from the original people, clothing, and scene while preserving the spatial relationships during the two-person exchange, the midair flip, the camera pulling back, and the back-to-back sword sheathing.
Motion material:
- Character material:
This set of materials tests both character replacement and the two-person relationship. The original video features two live-action swordsmen, and the target final video changes them into a panda and a tiger clashing in a bamboo forest after snow. If a live-action face, the original clothing, the orange smoke, or the panda and tiger swapping identities after occlusion remains in the generated result, it will be immediately obvious.
After uploading the motion material to LibTV, we first use the deep motion capture feature to extract the motion information of the original characters.
After extraction is complete, the depth information video appears on the canvas as a node, and we can use it for the next video generation.
We use the depth information video together with the uploaded character materials to generate a new video.
Prompt: Only the panda swordsman and tiger swordsman from the character reference images appear in the frame. The action position on the left wearing a conical hat at the opening is permanently bound to the panda, and the action position of the swordsman with tied-back hair on the right at the opening is permanently bound to the tiger; they must never swap throughout. Strictly preserve the attack and defense order, sword move directions, sword swing trajectories, blocking timing, footwork, body turns, leaps, distance between the two, crossing occlusion, and camera movement from the reference video. The panda always maintains black-and-white fur, a dark blue narrow-sleeved martial arts outfit, and a wide-brimmed conical hat; the tiger always maintains orange-and-black stripes, an off-white martial arts outfit, and brown-red shoulder guards. Each of the two characters holds one undecorated Chinese straight sword, with a straight blade and stable length; they must not merge or disappear on collision. The scene is changed to a stone platform in a bamboo forest after snow, cold moonlight, thin mist, and a small amount of snow on the ground. 11.54 seconds, 16:9, cinematic realistic 3D animation, continuous single-shot movement. The species, clothing, and body type of the two characters remain consistent throughout, their feet stay in contact with the ground, and their limb structures are normal. No third character is added to the frame, and no subtitles, logos, or watermarks appear.
The characters and scene have been replaced, but the action skeleton of the original video is still there. The positions of the panda and the tiger have not been swapped: the close-quarters exchange, the midair flip, the camera pulling back, and the final back-to-back sword sheathing all have corresponding matches in the original video.
Some friends might ask, can't I just generate directly from the original video without using this? Then let's try generating directly from the original video and two character images, with all other conditions the same except that no depth information video is used.
The generated video is as follows:
The main problem with the direct reference group appears in high-speed actions: the tiger's narrow sleeves were stretched into an outline close to the original video's wide sleeves, and the face and body were correspondingly elongated. So the video generated this way still carries part of the original character's outer silhouette, which does not match a pure reference action.
Let's try another video with constantly changing scenes to see whether depth motion capture can eliminate the influence of the scene.
Original video material:
Replacement character material Character A:
Replacement character material Character B:
We use depth motion capture and the material characters to generate a new video.
Prompt:
Use the extracted depth motion video as the only reference for motion, positioning, camera movement, and spatial relationships, and use the reference images of Character A and Character B as the only appearance references. Completely replace the two real people in the original video with two lightweight cyber mechas of similar height and build.
Character A maintains a pearl-white main armor, cobalt-blue chest armor and limb armor, black flexible inner layer, cyan horizontal visor, and cyan light strips throughout; Character B maintains a highly saturated dark red main armor, matte black inner layer, orange V-shaped visor, and orange light strips throughout. Both mechas use lean, agile human proportions close to those of adult male martial arts performers, with similar height, shoulder width, and limb length, close-fitting shoulder armor, clear waist, hip, and limb joints, and no heavy armor.
Strictly replicate the close-quarters blocking, continuous punches and kicks, kicking, dodging, chasing, vaulting, and falling actions in the depth motion video, preserving the two characters' starting positions, relative distance, contact timing, occlusion order, direction of movement, camera cuts, shot size changes, and original rhythm. The actions are fast and powerful while maintaining natural bending of mechanical joints, stable contact of both feet with the ground, and hands that always retain a five-finger structure.
Change the scene to a futuristic industrial cargo bay: gray-black metal walls, narrow passages, metal railings, cargo boxes, and cold white overhead work lights. Follow the spatial layout and obstacle positions in the reference video so that the characters have correct occlusion and contact with the railings, ground, and cargo boxes. The overall style is a cinematic realistic sci-fi action film, with realistic metal materials, cold gray ambient light paired with restrained cyan-orange character lighting, and motion blur preserved in high-speed actions.
19.3 seconds, 16:9, 30 fps, preserving the original shot order and editing rhythm of the reference video. Only Character A and Character B appear in the frame, with no additional characters, weapons, or drones added, and no subtitles, logos, or watermarks.
The action really follows well. The opening close-quarters exchange, continuous kicks, chase, vaulting over the railing, and final fall all have corresponding matches in the original video; after the real people and warehouse are replaced with mechas and a futuristic cargo bay, the character distances and obstacle occlusions are not disrupted.
03. Smart Editing
In the group, friends often ask me, is there software for one-click talking-head editing?
I happened to see it while doing motion capture testing: LibTV released a smart editing feature.
So I gave LibTV a livestream video of the GPT 5.6 launch event to test it out. The event switched back and forth between character interviews, phone demos, and webpage screen recordings. LibTV needed to reorganize by viewpoints and functional modules, not just trim by silence and pauses.
The above is our material for this session. We click Smart Editing to generate a node.
After we enter the prompt, we don't need to do anything else. LibTV will generate the needed material itself and edit it according to the prompt.
Prompt: Please intelligently edit this approximately 10-minute English product launch video into a 2-minute 30-second to 3-minute horizontal-screen tech news video, keeping the aspect ratio at 16:9. Core theme: What updates ChatGPT has launched for office work and team collaboration, and how it combines chat history, webpages, spreadsheets, and PDFs to complete real work. Editing structure: 1. Use content near "we're also announcing three major product updates" from the original video directly in the first 8 seconds, and quickly intercut phone, ChatGPT webpage, Excel spreadsheet, and PDF demo footage. Delete pleasantries and character introductions. 2. Use about 20 seconds to explain the background: teams are already using ChatGPT for writing, information lookup, and work analysis. 3. Keep the demos of the web version, mobile version, and calling team materials, focusing on how ChatGPT obtains context and answers ad hoc work questions. 4. Keep the demo of the finance team updating the Excel forecast model and viewing forecast and drivers. 5. Keep the final process of handing spreadsheets, PDFs, and other files to ChatGPT for analysis and organizing them into reporting materials. 6. Use original video content at the end that reflects "precision and correctness" or the final delivered result, and do not add new conclusions detached from the original video. Deletion requirements: Delete repeated expressions, pleasantries, laughter, uninformative pauses, waiting for loading, repeated input, and overly long mouse operations. Voiceover must keep complete sentences and must not cut off in the middle of a word. Demo footage should correspond to the feature being explained, avoiding staying on character interviews while the audio is talking about spreadsheets. Waiting and input processes can be sped up to 1.5 to 2 times, while people speaking should remain at normal speed. Each shot should remain 3 to 6 seconds as much as possible, avoiding continuous rapid jump cuts. Visual requirements: Keep the original video's existing Chinese and English bilingual subtitles, and do not generate another duplicate subtitle layer. When spreadsheets, PDFs, and the ChatGPT interface are involved, keep the complete operation area, and do not crop out text, input boxes, and generated results. Use direct cuts or slight fades, and do not use transitions such as lens flares, rotations, or glitch styles. Slight camera push-ins can be added, but they must not affect subtitle and interface readability. Do not add material, product names, features, or data unrelated to the content.
In the end, we got a 3-minute 6-second finished video. LibTV captured several main pieces of content: calling team materials on mobile, the finance team updating the forecast model, and organizing spreadsheets and PDFs into reporting materials. The pleasantries, waiting, and repeated operations in the original video were deleted, and the demos were not edited into a mess.
The biggest beneficiaries of Smart Editing should be talking-head creators in tech, finance, emotional, and knowledge popularization niches.
Shooting takes ten minutes, but the real time-consuming part is often the rough cut afterward: editors need to watch the material several times, find stumbles, repetitions, verbal tics, and ineffective pauses one by one, and then decide which lines can be deleted and which content must be kept. A single video may take 1 to 2 hours just for the rough cut. If you want to maintain daily updates, almost all the time is spent on this step.
- A little tip
Beginners often forget what the keyboard shortcuts are. You can view all shortcuts directly below the screen.
04. Some sharing
In this round of testing, LibTV's two new features happen to address the most time-consuming parts of video production: deep motion capture reduces interference from the original actor and original scene on the generated result, making motion references easier to reuse; smart editing first completes material screening and builds a rough-cut structure, so creators can save their time for pacing, subtitles, and final checks.
Google Veo, Kling, Seedance, Hailuo, and Vidu continue to push on model improvement, while LibTV goes deep into handling materials, shots, revisions, and delivery.
LibTV's long-term advantage depends on specialized tools and workflow assets. Models can be integrated and will be replaced; the nodes, reference materials, Skills, shot revision records, and team collaboration methods that creators save are more likely to create switching costs. Deep motion capture is only one feature, yet it is well suited to validating this product path.
AI video tools now compete on character consistency, motion control, and finished-video efficiency. Generating a few seconds of good-looking footage is already common; whether it can be done continuously, according to requirements, and smoothly integrated into creators' workflows will affect whether people are willing to pay long term.
Original link:With LibTV's deep motion capture, everyone can recreate action blockbusters