What Is GaussianCity
GaussianCity is an efficient unbounded 3D city generation framework launched by the S-Lab team at Nanyang Technological University, based on 3D Gaussian Splatting (3D-GS) technology. It introduces a compact BEV-Point representation that keeps the scene's VRAM usage at a constant level, solving the problem of excessively high VRAM and storage requirements in large-scale scene generation. GaussianCity introduces a spatially aware BEV-Point Decoder that, based on a point serializer and point transformer, captures the structural and contextual features of BEV points to generate high-quality 3D Gaussian attributes. GaussianCity performs excellently in 3D city generation tasks from drone viewpoints and street viewpoints, achieving a 60x speedup compared to existing methods (such as CityDreamer), reaching new heights in both generation quality and efficiency.
GaussianCity's Main Features
- Efficient generation of large-scale 3D cities: Quickly generates realistic, unbounded urban environments, supporting diverse scenes from drone viewpoints to street viewpoints.
- Low VRAM and storage requirements: Based on a compact BEV-Point representation, it keeps VRAM and storage requirements at a low level, solving the VRAM bottleneck problem of traditional methods in large-scale scene generation.
- High-quality visual effects: With a spatially aware BEV-Point Decoder, it generates 3D cities with high detail and realism, supporting stylized editing and local modifications.
- Real-time rendering and interaction: Supports real-time rendering and interaction, quickly generating high-quality 3D city images, suitable for scenarios such as games, animation, and virtual reality.
GaussianCity's Technical Principles
- 3D Gaussian Splatting (3D-GS): Uses 3D Gaussian distributions to represent objects and structures in the scene, with GPU-accelerated rendering, achieving efficient 3D scene generation.
- BEV-Point representation: Decomposes point information in the 3D scene into positional attributes and style attributes, compressing them based on bird's-eye view (BEV) and a style lookup table, ensuring that VRAM usage does not grow with scene scale.
- Spatially aware BEV-Point decoder: Based on a point serializer and point transformer, it captures the structural and contextual information of BEV points, generates 3D Gaussian attributes, and achieves high-quality 3D scene rendering.
- Efficient rendering pipeline: Combines a positional encoder and a modulation MLP to generate 3D Gaussian attributes, and uses a Gaussian rasterizer to render the attributes into the final image.
GaussianCity's Project Address
- GitHub repository：https://github.com/hzxie/GaussianCity
- HuggingFace model library：https://huggingface.co/spaces/hzxie/gaussian-city
- arXiv technical paper：https://arxiv.org/pdf/2406.06526
GaussianCity's Application Scenarios
- Games and virtual reality (VR): Quickly generates realistic virtual cities, supports real-time rendering and interaction, and reduces development costs and time.
- Animation and film/TV production: Efficiently generates complex urban scenes, supports stylized rendering, and improves production efficiency.
- Urban planning and architectural design: Quickly generates city layouts and architectural backgrounds, assisting visualization for planning and design.
- Autonomous driving and traffic simulation: Generates realistic traffic scenes for algorithm testing and traffic flow analysis.
- Geographic information systems (GIS): Quickly converts geographic data into 3D visualization scenes, supporting urban modeling and analysis.