stable-diffusion-videos is an open-source tool that uses the Stable Diffusion model to generate videos. Its core purpose is to achieve smooth morphing between different text prompts by exploring and interpolating in latent space, thereby creating dynamic videos. For example, it can generate a morphing video between "blueberry spaghetti" and "strawberry spaghetti".
Main capabilities include: generating videos based on text prompts; supporting direct execution in Colab; providing example scripts (located in the examples folder); adding a new music video feature, where by providing an audio file path, the audio will affect the interpolation rate, making the video move with the beat.
Usage: users can try it in Colab or refer to the example scripts. The project is based on a script shared by @karpathy and has been modified to form the current repository. For Apple M1 architecture, torch.float32 must be used because torch.float16 is unavailable on MPS.
Applicable scenarios: suitable for creative workers, researchers, or enthusiasts who need to generate morphing videos between text prompts. Limitations: no other detailed information is currently provided.
