OmniSVG - Fudan University and StepFun launch an end-to-end multimodal vector graphics generation model

What is OmniSVG

OmniSVG is the world's first end-to-end multimodal SVG (Scalable Vector Graphics) generation model jointly developed by Fudan University and StepFun. Built on a pre-trained vision-language model (VLM), through an innovative SVG tokenization method, it parameterizes SVG commands and coordinate parameters into discrete tokens, achieving the decoupling of structural logic and geometric details. This enables OmniSVG to efficiently generate diverse high-quality SVG graphics ranging from simple icons to complex anime characters.

Main Features of OmniSVG

• Multimodal Generation: OmniSVG is the first end-to-end multimodal SVG generation model, capable of generating high-quality SVG graphics based on text descriptions, image references, or character references. It can generate diverse graphics ranging from simple icons to complex anime characters.
• Efficient Generation and Training: Built on the pre-trained vision-language model (VLM) Qwen-VL, OmniSVG parameterizes SVG commands and coordinate parameters into discrete tokens through an innovative SVG tokenization method, separating structural logic from geometric details during training. This improves training efficiency by more than 3 times compared to traditional methods, can handle sequences of up to 30,000 tokens, and supports generating complex SVGs with rich details.
• Dataset and Evaluation: The OmniSVG team released the MMSVG-2M dataset, containing 2 million SVG resources with multimodal annotations, covering three major subsets: icons, illustrations, and characters. They proposed a standardized evaluation protocol, MMSVG-Bench, for testing the performance of conditional SVG generation tasks.
• Editability and Practicality: The generated SVG files have unlimited scalability and full editability, and can be seamlessly integrated into professional design workflows, such as tools like Adobe Illustrator, improving the practicality of AI-generated graphics in fields such as graphic design and web development.

Technical Principles of OmniSVG

• Based on a Pre-trained Vision-Language Model (VLM): OmniSVG is built on the pre-trained vision-language model Qwen-VL. The model can deeply integrate image and text information, providing a powerful foundation for multimodal generation.
• SVG Tokenization Method: OmniSVG innovatively parameterizes SVG commands and coordinates into discrete tokens, processing SVG generation in a manner similar to natural language processing. This improves training efficiency while retaining the ability to generate complex SVG structures.
• End-to-End Multimodal Generation Framework: OmniSVG supports directly generating SVG graphics from multiple input methods such as text descriptions, image references, or character references. This end-to-end generation framework can generate vector graphics with rich colors and vivid details, overcoming many limitations of traditional methods.
• Efficient Training and Long Sequence Processing: Compared with traditional methods, OmniSVG's training speed is improved by more than 3 times, and it can process sequences up to 30,000 tokens long. This enables it to generate complex SVG graphics containing rich details.

OmniSVG Project Address

• Project Official Website：https://omnisvg.github.io/
• Github Repository：https://github.com/OmniSVG
• HuggingFace Model Library：https://huggingface.co/OmniSVG
• arXiv Technical Paper：https://arxiv.org/pdf/2504.06263

Application Scenarios of OmniSVG

• Brand Icon Design: OmniSVG can quickly generate brand icons based on text descriptions, so designers do not need to draw from scratch, greatly reducing manual design time.
• Web Development: In web development, icons are indispensable elements. OmniSVG can generate vector icons based on text descriptions or image references, and the icons can be scaled losslessly, making them suitable for a variety of resolutions from mobile devices to 4K displays.
• Character and Scene Design: In game development, OmniSVG can be used to generate graphic assets such as game characters and scenes, adding a unique artistic style to games.
• Dynamic Character Generation: Based on a character reference, OmniSVG can generate vector graphics that preserve the same character features but with different poses or scenes.
• Rapid Prototyping: Content creators can use OmniSVG to quickly generate prototypes of icons, illustrations, or character graphics, accelerating the creative process.