What is DDColor
DDColor is an open-source AI image colorization framework launched by researchers from Alibaba DAMO Academy. It can colorize black and white photos into full-color images with one click. This method achieves automatic colorization of grayscale images by using a dual-decoder architecture (a pixel decoder and a color decoder), making the generated color images more realistic and vivid. It aims to solve the problems of multimodal uncertainty and high ill-posedness that exist in traditional image colorization methods.
How DDColor Works
The working principle of DDColor is based on an end-to-end deep learning model, which implements the image colorization process through two main decoder components.
The following is the core workflow of DDColor:
- Feature Extraction: First, a pretrained image classification model (such as ConvNeXt) is used as an encoder to extract high-level semantic features from the input grayscale image. These features contain the image's structure, texture, and object information.
- Pixel Decoder: The features extracted by the encoder are fed into the pixel decoder, which consists of a series of upsampling layers that gradually restore the spatial resolution of the image. Each upsampling layer is connected to the corresponding layer of the encoder through a shortcut connection, so as to preserve detail information while restoring the spatial structure.
- Color Decoder: The color decoder receives multi-scale visual features from the pixel decoder and generates color queries. These color queries are learned and are used to represent the colors of different regions in the image. The color decoder matches the color queries with the image features through a cross-attention mechanism, thereby generating colors that match the image content.
- Cross-Attention and Self-Attention Mechanisms: In the color decoder, the cross-attention layer is used to establish the association between color queries and image features, while the self-attention layer further refines these color queries so that they more accurately reflect the semantic content of the image.
- Color Richness Loss: To enhance the color richness of the generated image, DDColor introduces a color richness loss function based on the standard deviation and mean of the color planes. This loss function encourages the model to generate more colorful and vivid images.
- Fusion and Output: Finally, the outputs of the pixel decoder and the color decoder are combined through a fusion module to generate the final color image. This fusion process is implemented through a simple dot product operation, and then a 1×1 convolution layer generates the final AB (hue and saturation) channels.
- Training and Optimization: During training, the entire network optimizes the model by minimizing pixel loss, perceptual loss, adversarial loss, and color richness loss. These loss functions work together to ensure that the generated image is visually realistic while maintaining semantic consistency.
DDColor Official Entrances
- Official GitHub Project:https://github.com/piddnad/DDColor
- ModelScope Running Address:https://www.modelscope.cn/models/iic/cv_ddcolor_image-colorization/summary
- Replicate Running Address:https://replicate.com/piddnad/ddcolor
How to Use DDColor
- Visit DDColor's ModelScope MaaS community or Replicate running address
- Upload your black and white image or select a sample photo
- Click to run the test and wait for the image to be colorized