DistriFusion - A Distributed Parallel Inference Framework for High-Resolution Diffusion Models

What is DistriFusion

DistriFusion is a distributed parallel inference framework for high-resolution diffusion models. Through distributed parallel inference, it significantly accelerates the process of diffusion models generating high-resolution images on multiple GPUs. DistriFusion uses patch parallelism to split and assign images to different devices, achieving up to 6x inference speedup without additional training while maintaining image quality. The implementation of DistriFusion provides a new benchmark for AI content creation and parallel computing research.

Main Features of DistriFusion

• Distributed Parallel Inference: Executes the inference process of diffusion models in parallel on multiple GPUs to improve the speed of image generation.
• Image Segmentation: Splits high-resolution images into multiple small patches, each processed independently to achieve parallelization.
• No Additional Training Required: As a training-free algorithm, DistriFusion can be directly applied to existing diffusion models, such as Stable Diffusion XL.
• Maintains Image Quality: While accelerating image generation, it maintains the high quality of generated images through optimization techniques.
• Asynchronous Communication: Supports asynchronous data exchange, reducing delays caused by communication overhead.

Technical Principles of DistriFusion

• Patch Parallelism: DistriFusion splits the input image into multiple patches, each of which can be processed independently on different GPUs, thereby achieving parallelization.
• Asynchronous Communication: During parallel processing, DistriFusion adopts an asynchronous communication mechanism that supports data exchange between GPUs without blocking the computation process, thereby reducing waiting time and improving efficiency.
• Leveraging the Sequential Nature of the Diffusion Process: DistriFusion observes that the inputs of adjacent steps in diffusion models are highly similar, so it proposes a method that reuses the feature maps from the previous time step to provide contextual information for the current step.
• Shifted Patch Parallelism: By slightly shifting patches at each time step, it simulates interactions between patches without requiring explicit global communication.
• Pipelined Computation: The design of DistriFusion allows the computation process to be pipelined, with different GPUs able to work on different time steps simultaneously, further improving processing speed.
• No Sacrifice in Image Quality: DistriFusion greatly accelerates the image generation process, and through carefully designed algorithms ensures that the quality of generated images does not degrade.
• Applicable to Various Diffusion Models: DistriFusion is not limited to a specific diffusion model and can be applied to various existing diffusion models, such as Stable Diffusion XL, to achieve acceleration.

DistriFusion Project Address

Application Scenarios of DistriFusion

• AI Art Creation: In the art field, DistriFusion can quickly generate high-quality images, helping artists and designers realize creative ideas.
• Game and Film Production: In the production of visual effects for games and films, DistriFusion can accelerate the rendering process and shorten production cycles.
• Virtual Reality (VR) and Augmented Reality (AR): In VR and AR applications, DistriFusion can quickly generate realistic 3D environments and scenes.
• Data Visualization: In the field of data analysis, DistriFusion can be used to generate complex visualization images, helping users understand data more intuitively.
• Advertising and Marketing: DistriFusion can be used to quickly generate attractive advertising images and marketing materials, improving the appeal and effectiveness of advertisements.