Mubert-Text-to-Music is an open-source repository containing a Google Colab notebook that demonstrates how to implement prompt-based music generation through the Mubert API.
Purpose: This demo is intended to showcase the capability of generating music from text prompts and to provide a simple interface where users can generate music by entering prompts and duration. In addition, there is a demo combined with Deforum Stable Diffusion for generating instant prompt music videos.
Main capabilities:
- Generate music based on text prompts and a specified duration.
- Generate a unique music combination for each request, with an extremely low probability of repetition.
- Music is created in real time, not selected from a finished track library.
- Support generating loop tracks or tracks with a beginning and an end.
- Combine with Stable Diffusion to generate music videos.
How to use: Users can run the provided notebook through Google Colab. In the notebook, users can set parameters such as the prompt and duration, and then call the Mubert API to generate music. For music video generation, use the Stable Diffusion + Mubert notebook.
Technical principles: Both the input prompt and the Mubert API tags are encoded as latent space vectors of a transformer neural network, then the closest tag vector is selected for each prompt, and the corresponding tag is sent to the Mubert API for music generation. All sounds (such as independent loops for bass, lead, etc.) are created by musicians and sound designers, and are not synthesized by a neural network. Mubert's proprietary technology is responsible for analyzing, selecting relevant sounds, and constructing the arrangement and composition.
Applicable scenarios:
- Generate background music synchronized with images and videos.
- Rapidly prototype music ideas.
- Explore text-to-music generation technology.
Limitations:
- The custom license allows free use of the music with attribution, for synchronization with images and videos, but it must not be published as your own work on DSPs (such as Spotify, Apple Music, Deezer).
- Mention @mubertapp and the #mubert hashtag.
- For commercial licensing, contact Mubert.
- This demo is the first step toward more complex and accurate generation algorithms, but it requires time and resources.
All music is generated by the Mubert API. For more information, please visit www.mubert.com/.
