What is PixelFlow
PixelFlow is jointly launched by the University of Hong Kong and Adobeimage generationmodel, supporting direct image generation in pixel space. PixelFlow is based on efficient cascaded flow modeling, progressively increasing from low resolution to high resolution, reducing computational cost. PixelFlow achieves an FID score of 1.98 in the 256×256 ImageNet class-conditional image generation task, demonstrating excellent image quality and semantic control capabilities. PixelFlow also performs well in text-to-image generation tasks, supporting the generation of high-quality images that are highly consistent with text descriptions. PixelFlow's end-to-end trainability and efficient multi-scale generation strategy provide new research directions for next-generation visual generation models.
PixelFlow's main features
- High-quality image generation: Supports the generation of high-resolution, high-quality images.
- Class-conditional image generation: Generates corresponding images based on given class labels.
- Text-to-image generation: Generates images matching text descriptions, supporting complex semantic understanding and visual representation.
PixelFlow's technical principles
- Flow matching: Flow matching is a generative model technique that progressively transforms samples from a prior distribution (such as a standard normal distribution) into samples from the target data distribution based on a series of linear paths. During training, training samples are constructed based on linear interpolation, and the model is trained to predict the transformation velocity from intermediate samples to real data samples.
- Multi-scale generation: Gradually increases image resolution based on a multi-stage denoising process. Each stage starts from a lower-resolution noisy image and gradually denoises to increase resolution until the target resolution is reached. The method of gradually increasing resolution avoids performing all denoising steps at full resolution, significantly reducing computational cost.
- Transformer architecture：
- Patchify: Converts the spatial representation of the input image into a 1D sequence of tokens.
- RoPE（Rotary Position Embedding）: Replaces the original sine-cosine positional encoding with RoPE to better handle different image resolutions.
- Resolution embedding: Introduces additional resolution embeddings to distinguish different resolutions.
- Text-to-image generation: Introduces cross-attention layers in each Transformer block to align visual features with text input.
- End-to-end training: Trains directly in pixel space based on a unified parameter set, without the need for a pretrained VAE or other auxiliary networks. During training, the model uniformly samples training samples from all resolution stages and uses sequence packing technology for joint training, improving training efficiency and model scalability.
- Efficient inference strategy: During inference, PixelFlow starts from Gaussian noise at the lowest resolution and gradually denoises and increases resolution until the target resolution is reached. Supports multiple ODE solvers (such as Euler and Dopri5), allowing different solvers to be selected as needed to balance speed and generation quality.
PixelFlow's project address
- GitHub repository：https://github.com/ShoufaChen/PixelFlow
- arXiv technical paper：https://arxiv.org/pdf/2504.07963
- Online experience Demo：https://huggingface.co/spaces/ShoufaChen/PixelFlow
PixelFlow's application scenarios
- Art and design: Generates creative paintings, graphic design elements, and virtual characters.
- Content creation: Assists in video production, game development, and social media content creation.
- Education and Research: As a teaching tool, it helps understand complex concepts and assists scientific research visualization.
- Business and Marketing: Generate product design prototypes, advertising images, and brand promotion content.
- Entertainment and Interaction: Used in interactive stories, VR/AR content generation, and personalized image customization.