InternSVG - A Unified SVG Modeling Suite Launched by Shanghai AI Lab

What is InternSVG

InternSVG is a comprehensive "data-evaluation-model" suite for unified SVG modeling launched by Shanghai Artificial Intelligence Laboratory and others, comprising three major components: the SAgoGe dataset with over 16 million samples, the standardized SArena evaluation benchmark, and a unified multimodal large model based on InternVL3-8B. Through SVG-specific Tokens and two-stage training, InternSVG achieves unified modeling of understanding, editing, and generation tasks for icons, illustrations, chemical structures, and animations, significantly surpassing existing methods.

Main Features of InternSVG

• Semantic Understanding: InternSVG can parse the semantics and structure of SVG code, support generating detailed descriptions and answering multiple-choice questions, and accurately identify graphic content and attributes.
• Instruction Editing: Supports 10 editing operations, covering low-level edits such as color modification and geometric transformation, as well as high-level edits such as semantic color replacement and style transfer.
• Cross-Modal Generation: Supports generating static SVG (icons, illustrations, chemical structure formulas) from text or images, as well as generating vector animations from text or video.

Technical Principles of InternSVG

• Architectural Foundation: Built on the ViT-MLP-LLM paradigm based on InternVL3-8B, using InternViT-300M as the visual encoder to process input images or videos, and connecting through an MLP projection layer to the Qwen2.5-7B language model for sequence modeling.
• SVG-Specific Tokens: More than 200 special Tokens are designed for SVG syntax, covering 55 core tags, 42 attributes, and value ranges, compressing sequence length by 30-50% and effectively alleviating the context pressure of long-sequence modeling.
• Subword Embedding Initialization: New Tokens are decomposed into pretrained subwords and their embeddings are averaged as initial values, preserving semantic prior knowledge, reducing training loss by about 40%, and significantly accelerating convergence.
• Two-Stage Progressive Training: The first stage trains on structurally simple icon and chemical data to establish basic syntax awareness, and the second stage introduces long-sequence illustration and complex animation data, gradually improving the model's ability to handle complex structures through a curriculum learning strategy.
• Unified Task Modeling: Through a shared Transformer architecture, it simultaneously handles the three types of tasks: understanding, editing, and generation, enabling positive transfer of cross-task knowledge, avoiding separate model training for single tasks, and significantly improving parameter efficiency and generalization ability.

How to Use InternSVG

• Environment Preparation: After cloning the repository, create a Python 3.9 virtual environment and install the dependency packages; if evaluation is needed, download the ViCLIP checkpoint.
• Model Deployment: Download the InternSVG-8B model weights from HuggingFace, and use LMDeploy to start the API service to support multi-GPU parallel inference.
• Invoking Inference: Send requests through the standard OpenAI API format, supporting input text or images to generate SVG, or input SVG code for understanding and editing.
• Custom Training: Prepare the dataset and run the script that adds special Tokens, then sequentially execute the first-stage (simple data) and second-stage (full data) fine-tuning training.
• Model Evaluation: Download the SArena benchmark dataset, perform inference on model outputs, and calculate various evaluation metrics to verify performance.

InternSVG Project Address

• GitHub Repository：https://github.com/hmwang2002/InternSVG
• HuggingFace Model Library：https://huggingface.co/InternSVG/InternSVG-8B
• arXiv technical paper：https://arxiv.org/pdf/2510.11341

Key information and usage requirements for InternSVG

• Project positioning: InternSVG is a unified SVG modeling comprehensive suite launched by Shanghai AI Laboratory in collaboration with Shanghai Jiao Tong University, Nanjing University, and other institutions, and has been accepted by ICLR 2026.
• Core components: It includes three core components—the SAgoge multimodal dataset with over 16 million samples (covering four major domains: icons, illustrations, chemical structure formulas, and vector animation), the standardized SArena comprehensive evaluation benchmark (providing unified task definitions and evaluation metrics), and the InternSVG-8B unified multimodal large model based on InternVL3-8B.
• Hardware environment: Inference requires at least a single GPU (multi-GPU deployment is recommended to improve throughput), and training requires 96 NVIDIA A800 GPUs or equivalent computing power.
  Software dependencies: Python 3.9; PyTorch, Transformers, LMDeploy (for service deployment), and LLaMA-Factory (for training) need to be installed.

InternSVG's core advantages

• Unified modeling across all tasks: It breaks the limitations of traditional SVG understanding, editing, and generation tasks being isolated from one another, enabling positive cross-task knowledge transfer through a single model architecture, avoiding repeated model training for different tasks, and significantly improving parameter efficiency and generalization ability.
• Ultra-large-scale data foundation: Relying on SAgoge, the largest SVG multimodal dataset to date (over 16 million samples), it covers four high-value domains: icons, long-sequence illustrations, chemical structure formulas, and vector animation, providing the model with comprehensive training support ranging from simple static graphics to complex dynamic sequences.
• Professional technical optimization: It innovatively introduces more than 200 SVG-specific special tokens and a subword embedding initialization strategy, combined with two-stage progressive training (from simple icons to complex animations), effectively compressing sequence length by 30-50%, accelerating convergence, and significantly reducing the difficulty of long-sequence modeling.
• Comprehensive performance leadership: In the SArena standardized benchmark tests, compared with the strongest proprietary models such as Claude-4-Sonnet and GPT-4o, understanding accuracy improves by about 11%, editing task PSNR improves by about 34%, and generation task FID decreases by about 56%, achieving significant superiority across all tasks and domains.
• Precision in specialized domains: In specialized scenarios with extremely high requirements for topological accuracy and symbolic standardization, such as chemical organic structure formula generation, its restoration accuracy for bond angles, atom annotations, and ring structures is far ahead of existing baselines, meeting the high-precision needs of scientific visualization.

Comparison of InternSVG with similar competitors

Application scenarios of InternSVG

• Digital design creation: Designers can quickly generate high-quality vector icons and illustration assets through text descriptions or reference images, and support fine-grained editing such as color adjustment and style transfer based on natural language instructions, significantly improving UI/UX design efficiency.
• Scientific visualization: In scientific research fields such as chemistry and biology, researchers can automatically convert molecule names or structure images into precise vector chemical structure formulas, meeting the strict requirements of paper illustrations and textbook publishing for graphic standardization and editability.
• Intelligent content understanding: Automatically parses the semantic content and geometric structure of SVG icons or illustrations, supporting visual impairment assistance, image retrieval, and intelligent question answering and knowledge extraction based on graphic content.
• Dynamic media production: Generates vector animations (such as loading animations and icon motion effects) based on text scripts or reference videos, outputting lightweight and infinitely scalable animation assets that adapt to multi-terminal display requirements.
• Education and technical documentation: InternSVG supports real-time adjustment of graphic elements through natural language instructions, lowering the threshold for non-design professionals to create high-quality vector teaching content.