What is Bolt3D
Bolt3D is a new 3D scene generation technology jointly launched by Google Research, the University of Oxford VGG team, and Google DeepMind. It is a latent diffusion model that can sample a 3D scene representation directly from one or more images on a single GPU in less than seven seconds. On an NVIDIA H100 graphics processing unit, Bolt3D takes only 6.25 seconds to process photos into a complete three-dimensional scene.
Bolt3D's main features
- Fast 3D scene generation: Bolt3D is a feed-forward generation method that can sample a 3D scene representation directly from one or more input images. Generation is extremely fast, taking only 6.25 seconds on a single GPU.
- Multi-view input and generalization ability: It supports different numbers of input images, from single-view to multi-view, and can generate content for unobserved regions, with good generalization ability.
- High-fidelity 3D scene representation: It is based on Gaussian Splatting technology to store data, constructing a three-dimensional scene through three-dimensional Gaussian functions arranged in a two-dimensional grid. Each function records position, color, transparency, and spatial information, and the generated 3D scene is of high quality.
- Real-time interaction and applications: Users can view and render the generated 3D scene in real time in a browser, with broad application prospects, such as game development, virtual reality, augmented reality, architectural design, film and television production, and other fields.
Bolt3D's technical principles
- Geometric multi-view latent diffusion model: A multi-view latent diffusion model was trained to jointly model images and 3D point maps. The model takes one or more images and their camera poses as input and learns to capture the joint distribution of the target image, target point map, and source view point map.
- Geometric VAE: A geometric VAE was trained to jointly encode the point map of a view and the camera ray map into a geometric latent feature. The model is optimized by minimizing a combination of the standard VAE objective and a specific geometric loss, and can compress point maps with high precision.
- Gaussian head model: Given the camera and the generated image and point map, a multi-view feed-forward Gaussian head model was trained to output the refined color, opacity, and covariance matrix of the 3D Gaussians stored in the splat image.
- Large-scale multi-view consistent dataset: To train Bolt3D, a large-scale multi-view consistent 3D geometry and appearance dataset was created, generated by applying state-of-the-art dense reconstruction techniques to existing multi-view image datasets.
- Three-stage training process: A three-stage training process is adopted. First, the Geometry Variational Autoencoder (Geometry VAE) is trained, then the Gaussian head model is trained, and finally the latent diffusion model is trained.
Bolt3D's project address
- Project official website：https://szymanowiczs.github.io/bolt3d
- arXiv technical paper：https://arxiv.org/pdf/2503.14445
Bolt3D's application scenarios
- Game development: Quickly generate 3D scenes in games, reducing development time and cost.
- Virtual reality and augmented reality: Provide real-time 3D scene generation for VR and AR applications, improving the user experience.
- Architectural design: Quickly generate 3D models of buildings, facilitating design and presentation.
- Film and television production: Used for special effects production in films and TV series, quickly generating complex 3D scenes.