DressCode - A 3D clothing generation framework launched by ShanghaiTech University

What is DressCode

DressCode is a 3D clothing generation framework jointly launched by ShanghaiTech University, the University of Pennsylvania, and Deemos Technology. It supports users in automatically generating 3D clothing models of various styles and materials through text descriptions. Based on the core module SewingGPT, DressCode can understand text prompts and convert them into detailed cutting patterns, and then combine physically based rendering technology to create realistic clothing effects.

Main features of DressCode

1. Text-driven clothing generation: Users input text descriptions, and the system automatically generates the corresponding 3D clothing models.
2. Material and texture simulation: Generate different materials such as silk and lace based on text prompts, and simulate realistic lighting effects.
3. Semantic understanding and pattern generation: The SewingGPT module parses text and generates a sequence of cutting pattern tokens.
4. Physically based rendering: Advanced cloth dynamics algorithms simulate the draping and dynamic effects of real clothing.

Technical principles of DressCode

• Natural language processing (NLP): DressCode uses advanced NLP technology to parse and understand users' text input, capturing key features and style requirements in clothing descriptions.
• Sequence generation model: As the core component, SewingGPT adopts a Transformer-based decoder architecture to convert text descriptions into serialized representations (token sequences) of clothing cutting patterns.
• Quantization and dequantization: The parameters of clothing patterns are converted into tokens through a quantization process, and after the generation process, dequantization is performed to reconstruct the patterns in 3D space.
• Conditional generation: Using text conditional embeddings and cross-attention mechanisms, SewingGPT can generate clothing patterns that match the descriptions based on text prompts.
• Physically based rendering (PBR): DressCode uses PBR technology to generate realistic texture and material effects for clothing, simulating the reflection and refraction characteristics of different fabrics under lighting.
• Cloth dynamics simulation: Integrates advanced cloth simulation algorithms to ensure that clothing exhibits realistic physical behavior in virtual environments, such as wrinkles and swaying.

DressCode project address

Application scenarios of DressCode

• Fashion design: Designers can quickly generate clothing prototypes through text descriptions, accelerating the design process and enabling rapid iteration.
• Virtual try-on: E-commerce platforms and fashion brands can use DressCode to provide virtual try-on services, allowing consumers to experience the wearing effect of clothing online.
• Game development: Game designers can use DressCode to quickly generate clothing for game characters, enriching the appearance options in games.
• Film and television production: During the costume preparation stage of movies and TV series, DressCode can assist designers in quickly generating clothing concept art based on script descriptions.