Today I'm sharing something suitable for making storybooks.
Making a storybook normally involves such a tedious process: splitting storyboards, generating images one by one, handling subtitles, creating black-and-white-to-color animations, then setting transitions and export parameters. The workflow is very fragmented.
Today this GitHub project called story-to-handdrawn-video integrates the entire storybook-making process into one, and a single prompt can produce a finished piece.
I ran one and added sound with CapCut, and the result is really quite stunning:
01. Project Hands-On
case 1 Osmanthus Postcard
Prompt:
Use $story-to-handdrawn-video to turn the story below into a hand-drawn diary comic animation.
Task settings:
- Title: "Osmanthus Postcard"
- Use the default Codex Image2 image generation
- Use the project font to generate subtitles
- Use cut transitions
- First complete image generation, asset import, and preview rendering
- Output a 720×960, H.264 silent preview
- Keep the original story text, with each complete sentence as one storyboard
- Illustrations keep a pure white background, black hand-drawn lines, and low-saturation crayon colors
- All characters and objects must appear completely within the safe area
- No need to add voiceover, music, dialogue bubbles, or in-frame text
- After completion, copy the preview video as:
out/tests/guihua-postcard-preview.mp4
- Report the number of storyboards, video duration, and final file path
Character lock:
A Ning, 26-year-old woman, shoulder-length black hair, off-white short jacket, light blue long skirt, brown cloth shoes, carrying a dark green canvas bag. All scenes keep the same face shape, hairstyle, clothing colors, and body proportions.
White-haired elderly man, around 70 years old, short white hair, thin, wearing a gray Chinese-style jacket and dark blue trousers. The elderly man only enters the frame after he explicitly appears in the story.
Story text:
In the early morning, A Ning finds an unfinished postcard in an old book.
On the back of the postcard is only one sentence: When the osmanthus blooms, come home.
A Ning follows the old address to the south of the city and sees that the osmanthus by the courtyard gate has already bloomed.
A white-haired elderly man takes the postcard, his fingers resting on that line of text for a long time.
In the evening, when A Ning leaves, the elderly man hands her a small bag of osmanthus.
The protagonist's hairstyle, blue skirt, and green canvas bag are maintained very well; the old book, postcard, courtyard gate, and osmanthus tree form a clear scene progression. The subtitles are always readable, the characters and buildings are not cropped, and the final action of handing over the osmanthus is also very complete, bringing the story to a smooth close.
case 2 The Repaired Yellow Umbrella
Prompt:
Use $story-to-handdrawn-video to turn the story below into a hand-drawn diary comic animation.
Task settings:
- Title: "The Repaired Yellow Umbrella"
- Use the default Codex Image2
- Subtitles use the project font
- Use cut transitions
- Fully execute image generation, import, and preview rendering
- Keep the original text, with one complete sentence corresponding to one storyboard
- The yellow umbrella must always keep the same color, curved handle, and damaged spot
- Uncle Zhou's dark blue toolbox must keep the same appearance
– The illustration adopts a hand-drawn diary style on a white background, retaining plenty of negative space
– Watermarks, signboard text, speech bubbles, and extra characters are prohibited in the image
– After completion, copy the preview video as:
out/tests/yellow-umbrella-preview.mp4
– Report the number of storyboard panels, total duration, and file path
Character lock:
Uncle Zhou, a 55-year-old male with short gray hair, a square face, wearing a dark blue maintenance worker uniform and black cloth shoes, always carrying an old dark blue toolbox.
Lin Xi, a 20-year-old female with a black high ponytail, wearing a dark green knit sweater, beige trousers, and white canvas shoes, holding two books in her arms.
The two characters' face shapes, hairstyles, clothing colors, and body proportions must remain consistent.
Story text:
After the night shift ended, Uncle Zhou noticed a girl holding books still sitting in the bus stop.
The rain kept getting heavier, and one of the ribs of the yellow umbrella in the girl's hand suddenly broke.
Uncle Zhou found thin wire in his toolbox and, with his head lowered, wrapped the umbrella rib round after round.
The last bus arrived, and before getting on, the girl placed a tangerine beside the toolbox.
The next morning, Uncle Zhou opened the toolbox and saw a note under the tangerine that read Thank you.
Uncle Zhou, the girl, the dark blue toolbox, and the yellow umbrella remain stable, and the bus stop, rain lines, last bus, and early morning form a complete progression of time. The tangerine appears before the girl leaves and is still beside the toolbox the next day, and the story has no missing shots.
But the broken umbrella rib is not clearly depicted, and at first glance it still looks like a complete umbrella.
case 3 Spring in the Blue Cloth Bag
Prompt:
Use $story-to-handdrawn-video to turn the story below into a hand-drawn diary comic animation.
Task settings:
– Title: "Spring in the Blue Cloth Bag"
– Use the default Codex Image2
– Subtitles use the project font
– Use cut transitions
– First build a visual-plan JSON with two-digit scene numbering based on the story
– The visual-plan must specify Jiang He's age, clothing, location, actions, and the characters that need to appear in each scene
– Save the visual-plan to:
out/tests/mint-visual-plan.json
– Use that visual-plan to continue executing image generation, import, and preview rendering
– Keep the original text, with each sentence as one story beat
– The blue cloth bag, white flowerpot, and mint seeds need to maintain visual continuity
– Grandma only enters the frame in storyboard panels where she explicitly appears in the original text
– After completion, copy the preview as:
out/tests/mint-spring-preview.mp4
– Report the number of storyboard panels, total duration, visual-plan path, and video path
Character lock:
Jiang He has a round face, narrow elongated eyes, a small mole at the tail of the left eyebrow, and slightly curly black hair; these features remain consistent across different ages.
At age 8, Jiang He wears a light yellow sweater and dark blue overalls.
At age 16, Jiang He wears a white shirt and gray-blue trousers, carrying a brown schoolbag.
At age 26, Jiang He has shoulder-length slightly curly black hair and wears a light gray trench coat and dark green trousers.
Grandma is about 68 years old, with short silver-gray hair, wearing a dark red cardigan and a dark gray long skirt.
Story text:
When Jiang He was eight, every day she stood on a small stool and watered the mint on the windowsill.
Grandma sewed a small packet of mint seeds into a blue cloth bag and gave it to Jiang He.
At sixteen, Jiang He left the small town, taking only that blue cloth bag.
Ten years later, Jiang He dragged a suitcase back to the old house, where dust had already settled all over the windowsill.
Jiang He planted the seeds from the blue cloth bag into the original white flowerpot.
When spring came, a new mint plant grew in the flowerpot.
At ages 8, 16, and adulthood, Jiang He shows discernible age changes; the slightly wavy black hair and round face preserve a certain identity continuity, while the blue cloth bag, white flowerpot, and old windowsill form stable visual cues.
When adult Jiang He returns to the dust-covered old house, the empty flowerpot and suitcase effectively convey the passage of time; the new mint and blue cloth bag appearing together at the end gives the emotional beat an accurate landing point.
However, the small mole at the end of the eyebrow is almost invisible under the current art style and character size; such details are not suitable for carrying identity recognition.
case 4 The old keychain at the noodle shop
Prompt:
Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.
Task settings:
– Title: “The Old Keychain at the Noodle Shop”
– Use the default Codex Image2
– Use the project font for subtitles
– Use cut transitions
– Fully execute image generation, asset import, and preview rendering
– Preserve the original story text and dialogue wording
– Each complete sentence corresponds to one storyboard panel
– Dialogue is displayed only in the upper subtitle area; do not generate dialogue bubbles in the illustrations
– Express rain, wet clothes, and hot soup noodles with simple hand-drawn lines
– The old keychain needs to maintain the same round wooden appearance
– Avoid exaggerated crying and dramatic body movements
– After completion, copy the preview as:
out/tests/noodle-shop-preview.mp4
– Report the number of storyboard panels, total duration, and video path
Character lock:
The proprietress, a 48-year-old woman with short curly black hair, wearing a brick-red apron, an off-white shirt, and dark gray trousers.
The delivery rider, a 24-year-old man with short black hair, thin build, wearing a soaked light blue delivery raincoat and black trousers, holding a white helmet in his hand.
Keep character face shapes, clothing color schemes, and body proportions consistent across all scenes.
Story text:
Late at night, the noodle shop is about to close, and a delivery rider soaked through stands at the door searching through his pockets.
The proprietress brings out a bowl of hot soup noodles and says: I cooked too much noodles, don’t waste them.
The delivery rider eats the noodles with his head down, raindrops from his helmet falling one by one by his feet.
Before leaving, the delivery rider places an old keychain on the table and says: I’ll come redeem it tomorrow.
The next day, as soon as the proprietress opens the door, she sees a bag of hot soy milk hanging on the door handle.
The old keychain is still placed in the most conspicuous spot on the counter.
The atmosphere at the noodle shop entrance, the rain, the wet clothes, and the hot soup noodles are very good; the proprietress’s red apron and the delivery rider’s blue raincoat remain stable, and the sequence of actions—hot noodles, eating with head down, putting something down, opening the door the next day—is clear.
However, the old keychain looks like a brown round piece, more like a coin in shape.
case 5 The watch stopped at six ten
Prompt:
Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.
Task settings:
– Title: “The Watch Stopped at Six Ten”
– Use the default Codex Image2
– Subtitles use the project font
– Use cut transitions
– Keep the original story text; do not shorten sentences in advance
– First run plan mode and report the automatic split results
– After reporting, continue to complete image generation, asset import, and preview rendering without waiting for confirmation
– Each storyboard subtitle has at most three lines
– The blue collar, old watch, and brown puppy must remain consistent
– The white-haired woman only appears in scenes after the phone number is connected
– When finished, copy the preview as:
out/tests/watch-at-six-ten-preview.mp4
– Report the original number of sentences, the number of storyboards after splitting, the total duration, and the file paths
Character lock:
Uncle Xu, a 62-year-old man, short gray hair, round-framed glasses, wearing an old brown vest, light gray shirt, and dark trousers.
White-haired woman, about 65 years old, ear-length white hair, wearing a dark green coat and a black long skirt.
The puppy is a short-haired brown dog with its right ear folded down, wearing a blue collar. A round silver old watch hangs on the collar.
Story text:
In the evening, Uncle Xu, a watch repairer, closed the shop door. He had just taken two steps out when he heard a soft scratching sound from under the counter.
Uncle Xu bent down and saw a puppy wearing a blue collar. Hanging on the collar was an old watch stopped at 6:10.
Later, Uncle Xu called the phone number on the back of the watch. A white-haired woman answered the phone. After a few seconds of silence, she burst into tears.
Half an hour later, the woman held the puppy. Uncle Xu handed her the repaired watch, and the hands passed 6:10 again.
Uncle Xu's glasses, brown vest, and watch repair shop remain stable. The brown puppy and blue collar show no obvious drift. Closing the shop, discovering the puppy, making the phone call, the woman crying, and returning the watch—all story beats are present.
The number of shots after automatic splitting is reasonable, and long sentences are not crammed into the same subtitle.
Overall, story-to-handdrawn-video has a good art style and color palette, and the story line transitions smoothly, but if key items are too small, they are not very visible。
02. Project Introduction
story-to-handdrawn-video Turn Chinese story text or a set of ordered images into a 3:4 vertical-screen hand-drawn comic animation. The finished video uses a fixed visual grammar of a white background, handwritten subtitles, and black-and-white line art transitioning to color illustrations, and outputs a silent H.264 video, leaving voiceover and music for post-production.
Project address:https://github.com/gnipbao/story-to-handdrawn-video
The project is positioned as an Agent production workflow, covering story sentence segmentation, storyboard planning, drawing tasks, asset cropping, animation arrangement, and video export.
story-to-handdrawn-video It is suitable for family stories, children's stories, diary comics, and WeChat official account story videos with a fixed style. It is also suitable for quickly converting a batch of already-generated comic pages into silent assets with reveal animations.
The project currently only supports single-image storyboards, a fixed white-background hand-drawn style, a 3:4 canvas, and silent output. Lip sync, character actions, camera movement, voiceover synthesis, subtitle timelines, 9:16 finished videos, and multiple art styles still need further development.
story-to-handdrawn-video One beat per sentence, character references, black-and-white-to-color, and silent export are written as a repeatable executable workflow. The current version is suitable for very content production.
To install, simply give the GitHub link to an AI Agent and let it help you install this project.
When using it, everyone should preferably choose an AI Agent with an image generation model, so that a video can be generated in one go with a single prompt.
03. Dig a Little
Short videos and image-text content increasingly rely on stable, continuous visual production.
For individual creators, small studios, and content teams, finishing the story is only the first step; what really consumes time is the back-and-forth switching between storyboarding, image generation, subtitles, animation, and exporting.
story-to-handdrawn-video brings these scattered steps into a repeatable workflow. As long as the character settings, scene changes, and key props are clearly written in advance, the AI Agent can continue to complete storyboarding, illustration generation, black-and-white to color animation, and video exporting, quickly producing a version of a sample reel that can be watched directly.
This workflow is suitable for WeChat public account stories, children's picture books, family short films, diary comics, brand stories, and IP content testing. It can reduce software switching, shorten the draft trial cycle, and batch-verify story directions before deciding whether to invest in voiceover, editing, and fine animation.
story-to-handdrawn-video At this stage, it is more suitable for producing initial versions and silent materials; voiceover, music, and complex shots still need to be added in post-production.
Original link:GitHub low-star hidden gem, turn a story into a picture book in five minutes