What is SANA 1.5
SANA 1.5 is a new and efficient Linear Diffusion Transformer launched by NVIDIA in collaboration with MIT, Tsinghua University, Peking University, and other institutions, used fortext-to-image generationtask. Based on SANA 1.0, it introduces three major innovations: efficient training scaling, which uses a depth growth paradigm to scale the model from 1.6 billion parameters to 4.8 billion parameters, significantly reducing computational resources and incorporating an efficient 8-bit optimizer. Model depth pruning, which efficiently compresses the model by analyzing block importance, allowing flexible adjustment of model size under different computational budgets. Inference-time scaling, which uses repeated sampling and a Vision-Language Model (VLM)-based selection mechanism to enable small models to achieve the quality of large models at inference time.
Main Features of SANA 1.5
- Efficient Training Scaling: SANA 1.5 adopts a depth growth paradigm that can scale the model from 1.6 billion parameters to 4.8 billion parameters, significantly reducing computational resources.
- Model Depth Pruning: SANA 1.5 introduces a model compression technique based on block importance analysis, which can efficiently compress large models to any size with minimal quality loss. By analyzing the similarity patterns of inputs and outputs in the diffusion transformer, unimportant blocks are pruned, and model quality is quickly restored through fine-tuning.
- Inference-Time Scaling: SANA 1.5 proposes an inference-time scaling strategy that uses repeated sampling and a Vision-Language Model (VLM)-based selection mechanism to enable small models to achieve the quality of large models at inference time.
- Multilingual Support: SANA 1.5 supports multilingual text input, including Chinese, English, and emojis, suitable for global content creation and localized design.
- Open Source and Community Support: The code and pre-trained models of SANA 1.5 have been open-sourced, allowing researchers and developers to customize and extend them, further promoting its adoption in academic research and industrial applications.
- Inference Efficiency: Through the CAME-8bit optimizer, SANA 1.5 can fine-tune large-scale models on a single consumer-grade GPU, making high-quality image generation more efficient and accessible.
Performance Testing of SANA 1.5
- Model Growth: Through the model growth strategy, SANA 1.5 scales from 1.6 billion parameters to 4.8 billion parameters, improving the GenEval score from 0.66 to 0.72, close to the industry-leading Playground v3 (2.4 billion parameters) at 0.76, while reducing inference latency by 5.5 times.
- Model Pruning: Through depth pruning, SANA 1.5 can flexibly adjust model size under different computational budgets. For example, after pruning a 4.8 billion parameter model to 1.6 billion parameters and fine-tuning for 100 steps, the GenEval score reaches 0.672, surpassing the 0.664 of the SANA 1.0 1.6 billion parameter model.
- Inference Scaling: By generating multiple samples and selecting the best one based on VLM, the GenEval score of SANA 1.5 increases from 0.72 to 0.80, surpassing Playground v3's 0.76.
Project Address of SANA 1.5
- arXiv Technical Paper：https://arxiv.org/pdf/2501.18427
Application Scenarios of SANA 1.5
- Creative Design: SANA 1.5 can generate high-quality images based on text prompts, suitable for creative design fields such as advertising design, illustration creation, game art, etc.
- Teaching Assistance: Teachers can use SANA 1.5 to generate course-related images, helping students better understand abstract concepts.
- Film and Television Production: In film and television production, SANA 1.5 can generate concept art, scene design drawings, etc., helping directors and art directors quickly conceptualize and validate ideas.
- Engineering Design: Engineers can use SANA 1.5 to generate visual renderings of engineering designs, helping teams better understand design intent and optimize design solutions.
- Mobile Applications: Through model depth pruning and inference-time scaling, SANA 1.5 can run efficiently on mobile devices, providing real-time image generation capabilities for mobile applications.
- Content Moderation: Combined with safety check models (such as ShieldGemma-2B), SANA 1.5 can moderate user-input text before generating images, ensuring that the generated content meets safety standards and avoiding the generation of inappropriate content.