What is LBM
LBM (Latent Bridge Matching) is a novel image-to-image transformation framework launched by the Jasper Research team, based on constructing bridge matching in the latent space to achieve fast and efficient image transformation. LBM requires only a single inference step to complete the task and is suitable for various image transformation scenarios, such as object removal, relighting, depth and normal estimation, etc. LBM uses Brownian bridges to establish random paths between the source image and the target image, increasing sample diversity. The conditional framework enables controllable shadow generation and image relighting. LBM achieves or surpasses the best performance of existing methods in multiple tasks, demonstrating strong versatility and efficiency.
Main Features of LBM
- Object Removal: Remove specified objects and their related shadows from the image while maintaining the integrity of the background.
- Image Relighting: Relight the foreground object according to the given background or lighting conditions, removing existing shadows and reflections.
- Image Restoration:Convert degraded images into clean images, restoring the original quality of the image.
- Depth/Normal Map Generation: Convert the input image into a depth map or normal map for tasks such as 3D reconstruction.
- Controllable Shadow Generation: Generate shadows based on the position, color, and intensity of the light source, enhancing the realism of the image.
Technical Principles of LBM
- Latent Space Encoding: Encode the source image and target image into a low-dimensional latent space, reducing computational cost and improving the scalability of the model.
- Brownian Bridge: Construct a random path (Brownian bridge) in the latent space, connecting the latent representations of the source image and the target image. The randomness supports the model in generating diverse samples.
- Stochastic Differential Equation (SDE): Based on solving the stochastic differential equation (SDE), predict the latent representation along the path, achieving the transformation from the source image to the target image.
- Conditional Framework: By introducing additional conditional variables (such as lighting maps), LBM can achieve controllable image relighting and shadow generation.
- Pixel Loss: During training, use pixel loss (such as LPIPS) to optimize the model, ensuring the visual consistency between the generated image and the target image.
LBM Project Address
- Project Website：https://gojasper.github.io/latent-bridge-matching/
- GitHub Repository：https://github.com/gojasper/LBM
- arXiv Technical Paper：https://arxiv.org/pdf/2503.07535
- Online Experience Demo：https://huggingface.co/spaces/jasperai/LBM
LBM Application Scenarios
- Ordinary Users: Daily photo editing, removing unwanted objects, repairing old photos, adjusting lighting.
- Photography Enthusiasts: Post-processing, enhancing photo realism, adding or adjusting shadows.
- Graphic Designers: Creative design, generating depth/normal maps, quickly repairing and adjusting images.
- Video Editors: Video production, repairing video frames, adjusting object lighting and shadows.
- 3D Modelers: Generating depth/normal maps from photos to assist 3D modeling.