What is LaDeCo
LaDeCo is an automatic graphic design composition method jointly launched by Xi'an Jiaotong University and Microsoft Research, implemented by decomposing design tasks into hierarchical steps. LaDeCo performs layer planning on input design elements, assigning them to different semantic layers, such as background, base, image/logo, text, and decoration. Then, LaDeCo predicts layer by layer, generating the element attributes of each design layer, using the rendered images of previously generated layers as contextual information to guide the generation of subsequent layers. LaDeCo is based on large multimodal models (LMMs) to process multimodal inputs and supports design subtasks that require no task-specific training, such as resolution adjustment, element filling, and design variation.
Main Features of LaDeCo
- Layer Planning: Automatically assigns input multimodal design elements (such as images and text) to different semantic layers, for example, background, base, image/logo, text, and decoration layers.
- Hierarchical Design Generation: Based on the results of layer planning, progressively predicts and generates the element attributes of each layer to create a complete design composition.
- Resolution Adjustment: Adjusts the design according to different canvas sizes, making the design attractive on canvases of different sizes.
- Element Filling: Adds new elements to an existing design to enhance the design's appeal.
- Design Variation: Given the same input elements, creates multiple different designs, providing users with multiple choices.
Technical Principles of LaDeCo
- Large Multimodal Models (LMMs): Based on large multimodal models, understands multimodal context and generates cross-domain responses.
- Layer Planning Module: Based on pre-trained LMMs (e.g., GPT-4o), predicts the semantic labels of input elements, achieving automatic classification of elements into design layers.
- Hierarchical Generation Process: According to the results of layer planning, generates the attributes of design elements layer by layer, feeding back the rendered images of generated layers as contextual information to the model to guide the generation of subsequent layers.
- Visual Encoder and Projector: Used to encode element images and intermediate designs, generate image embeddings, and project them to match the hidden state dimensions required by the LMM backbone.
- Chain-of-Thought Reasoning: LaDeCo's hierarchical generation method embodies the concept of chain-of-thought reasoning, improving reasoning performance based on progressively generating and adjusting design layers.
Project Address of LaDeCo
- Project Official Website：elements2design.github.io
- arXiv Technical Paper：https://arxiv.org/pdf/2412.19712
Application Scenarios of LaDeCo
- Designers: Helps designers automatically complete graphic design composition tasks, improving design efficiency and quality.
- Researchers and Planners: In landscape change research, aesthetic evaluation, and visual impact assessment, enables researchers and planners to quickly and objectively calculate the proportion of visual elements in images, simplifying the evaluation process.
- Evaluators: Plays an important role in evaluating visual landscape elements, helping evaluators conduct more efficient visual element analysis.
- Developers: For developers, develop different applications based on LaDeCo.
- General Young Population: The application of LaDeCo in the field of automated graphic design appeals to people aged 19-35 who have a relatively high preference for creative content, sharing, music, short videos, games, fashion, and so on.