FaceChain-FACT: A Face Adapter for Human AIGC
FaceChain-FACT is a zero-shot face adapter designed to address the challenges caused by high facial detail in customized face image generation. The traditional FaceChain work integrates face information and generates customized portraits by training an individual's LoRA model, but this process is divided into two stages, training and inference, which increases user costs. FaceChain-FACT does not require training a face LoRA model and can generate customized portraits with only a single photo from the user.
Main capabilities and features:
- Zero-shot generation: no need to train the user's LoRA model, only a single photo is required.
- Speed improvement: compared with SOTA commercial applications, generation speed is accelerated by 100 times, reaching second-level image generation speed.
- Face feature extraction: integrates a Transformer-based face feature extractor with a structure similar to Stable Diffusion, enabling Stable Diffusion to better utilize face information.
- Fine-grained features: uses dense fine-grained features as face conditions, providing better character fidelity.
- Plugin compatibility: seamlessly compatible with ControlNet and LoRA plugins, plug-and-play.
- Diverse styles: by fusing various LoRA models from FaceChain, portraits in multiple styles can be generated.
Technical principles: The system uses a series of image preprocessing steps, including face segmentation, face cropping and alignment, hand detection, face quality screening, etc., to screen and obtain the training dataset. It uses a Transformer-based face feature extractor to extract features, and uses dense fine-grained features from the penultimate layer as face conditions. Stable Diffusion receives face conditions through FACT-Adapter and combines them with text embeddings to generate portrait images.
Usage and applicable scenarios: This page mainly presents the technical introduction and generation results of FaceChain-FACT, and does not provide specific installation commands, code, or usage steps. Applicable scenarios include personalized portrait generation, customized face image creation, etc. Compared with FaceChain, its advantages are that no training is required, it is fast, and only a single photo is needed.
Limitations: The page does not provide specific usage restrictions, platform support, pricing information, or detailed operation guide. In addition, the page also mentions that Deformable Neural Radiance Fields can createfree-viewpoint portraits (nerfies), but does not elaborate further.
