Spark 2.0 - World Labs' open-source web-based 3D Gaussian Splatting rendering engine

What is Spark 2.0

Spark 2.0 is an open-source web-based 3D Gaussian Splatting rendering engine from World Labs (founded by Fei-Fei Li), built on Three.js and WebGL2. Spark 2.0 pioneers LoD (Level of Detail) systems, progressive streaming, and virtual memory technology, enabling real-time rendering of large-scale 3D worlds with over 100 million splats in browsers on any device, including desktops, phones, and VR. It achieves smooth, progressive loading of scenes from coarse to fine, breaking through the scale and performance bottlenecks of traditional Web 3D rendering, making instant interaction and immersive, smooth browsing of massive, ultra-large scenes possible.

Main Features of Spark 2.0

• Ultra-Large-Scale Scene Rendering: Supports streaming and real-time rendering of ultra-large 3D worlds containing over 100 million splats in browsers on any device, including desktops, phones, and VR.
• Multi-Object Global Compositing: Renders multiple independent 3DGS objects simultaneously, automatically performing unified global depth sorting of all splats to ensure different objects blend correctly in space rather than simply overlapping.
• Continuous LoD (Level of Detail): By building a hierarchical LoD Splat Tree structure, it dynamically calculates the optimal tree cut based on camera distance and viewing angle, automatically displaying high precision up close and low precision at a distance, achieving smooth, jump-free transitions in detail.
• Progressive Streaming: Utilizes a proprietary .RAD file format, allowing scenes to immediately display a coarse version with 64K splats on first load, prioritizing download and refinement of visible area data based on viewing angle, delivering a smooth experience that refines while streaming.
• Virtual Memory Management: Allocates a fixed GPU memory pool for splats as a page table, automatically fetching and swapping data blocks from the network based on the user's real-time position in the scene, overcoming mobile device memory limits to access massive data.
• Foveated Rendering Optimization: Through Foveated Rendering technology, it automatically shifts the rendering budget toward the center of the field of view, reducing the level of detail at the screen edges and areas behind the camera, significantly improving frame rates without affecting subjective image quality.
• Real-Time Editing and Relighting: Supports real-time editing, color adjustment, and dynamic relighting of 3D Gaussian Splatting scenes directly in the browser, with no offline preprocessing required to see modification effects.
• Programmable Effects System: Provides a GLSL-based code interface and a visual shader graph node system, enabling developers to create dynamic animations, SDF geometry clipping, transparency adjustments, and other rich custom visual effects.

Technical Principles of Spark 2.0

• 3D Gaussian Splatting Fundamentals: Spark 2.0 uses 3D Gaussian Splatting technology, replacing traditional triangular mesh scene representation with millions of semi-transparent 3D ellipsoids (splats). Each splat is defined by position, scale, rotation, color, and opacity, and is rendered by the GPU blending in depth order to form the final image.
• Three-Step Rendering Pipeline: The system uses a "generate global list → global depth sorting → instanced drawing" architecture, transforming multi-object splats into a unified coordinate system and applying dynamic effects (animation, clipping, etc.). After GPU computes distances, CPU radix sorting determines render order, and finally a single draw call completes rasterization of all splats.
• Continuous LoD Splat Tree: To address the rendering pressure of billion-scale splats, Spark builds a hierarchical tree structure from the bottom up, merging child nodes into parent nodes to form a multi-resolution representation. At runtime, it dynamically computes the optimal "tree cut line" through a priority queue within a fixed budget (500,000 to 2.5 million splats), achieving a smooth transition from fine near and coarse far.
• Progressive Streaming: It uses the self-developed .RAD format to chunk data (each chunk 64K splats) and store it in columnar compressed form. The file header records a random access index, allowing the scene to be presented immediately in a 64K coarse version, and dynamically downloading refined data chunks via HTTP range requests based on the viewing angle, achieving progressive loading where viewing happens while downloading.
• Virtual Memory Management: It allocates a fixed GPU memory pool as a splat page table, dynamically determines the data chunks required for the current viewport in combination with the LoD tree traversal results, and automatically pulls new data from the network and replaces old data when the user moves, enabling the browser to access massive splats far beyond local memory limits.
• LoD Tree Generation Algorithm: It provides two training-free algorithms to build the hierarchical structure: Tiny-LoD (fast merging based on spatial grids, using sorting instead of hashing to optimize caching, suitable for real-time generation on the web) and Bhatt-LoD (high-quality merging based on the Bhattacharyya distance to measure Gaussian similarity, suitable for offline processing).

How to Use Spark 2.0

• Environment Setup: Install Spark 2.0 and Three.js in the project via CDN or npm, and ensure the target browser supports the WebGL2 standard.
• Basic Loading: For small-scale scenes, directly call the loadSplat method to load raw 3DGS files in PLY, SPZ, or Splat format and add them to the Three.js scene.
• Large Scene Preprocessing: For large-scale scenes with over 100 million splats, use the command-line tool spark build-lod to convert the raw files offline into the .RAD format that supports progressive streaming.
• Streaming Loading: In the code, call the loadRad method to load the converted file, and configure the splat rendering budget and foveated rendering parameters to control the level of detail and performance.
• Render Update: Continuously call the splat object's update method in the animation loop and pass in the current camera parameters, automatically completing depth sorting and LoD detail switching.
• Multi-Object Compositing: When loading multiple LoD objects at the same time, Spark automatically traverses the splat trees of all objects and uniformly allocates the global rendering budget, achieving seamless fusion.

Key Information and Usage Requirements for Spark 2.0

• Product Positioning: An open-source web-side 3D Gaussian Splatting (3DGS) rendering engine from World Labs (founded by Fei-Fei Li), built on Three.js and WebGL2, capable of rendering large-scale 3D scenes with over 100 million splats in real time in browsers on any device.
• Core Technologies: ① LoD Splat Tree (continuous level of detail, dynamically selecting rendering precision); ② Progressive streaming (.RAD format, presenting a coarse version first and then refining on demand); ③ Virtual memory (a fixed GPU memory pool dynamically swapping data chunks).
• Performance: Supports all platforms including desktop, iOS, Android, and VR, breaks through device memory limits, and delivers a smooth experience of interaction while downloading.
• Environment Requirements：
  • Supports WebGL2 modern browsers (Chrome, Safari, Firefox, Edge, etc.).
  • No WebGPU required, broader compatibility
• Hardware support: Desktop, mobile (iOS/Android), VR devices (Quest 3, Apple Vision Pro, etc.).

Core advantages of Spark 2.0

• Ultra-large-scale capacity: Breaking through the limitation that consumer-grade devices typically support only 1-5 million splats, it can render giant scenes of 100 million+ splats in real time.
• Instant streaming experience: Using the self-developed .RAD format and progressive transmission technology, scenes are presented instantly with a 64K rough version, with no need to wait for hundreds of MB of data to fully download.
• Seamless compatibility across all platforms: Built on WebGL2, it covers desktop, iOS, Android, and VR devices, with no need to worry about API support differences.
• Intelligent detail balance: The original continuous LoD Splat Tree system dynamically adjusts precision based on viewing distance and gaze point, with near objects detailed and far objects blurred, and smooth transitions without jumps.
• Multi-object global fusion: Supports loading multiple independent 3DGS objects simultaneously, automatically performing cross-object global depth sorting, solving the traditional renderer's "texture overlay" problem.
• Imperceptible memory expansion: Through a virtual memory page table mechanism, data blocks are automatically swapped in from the network, enabling the browser to access massive resources far beyond local GPU memory limits.

Spark 2.0 project addresses

• Project website：https://www.worldlabs.ai/blog/spark-2.0
• GitHub repository：https://github.com/sparkjsdev/spark/

Comparison of Spark 2.0 with similar competitors

Application scenarios of Spark 2.0

• Digital twins and urban governance: Used to build a city-level digital twin foundation, supporting real-time browsing in the browser of smart city 3D models containing tens of millions of splats, enabling remote inspection and management without deploying heavy clients.
• VR/AR immersive experiences: Used for large-space immersive roaming on VR/AR headsets, allowing smooth exploration of ultra-large-scale virtual cultural tourism and world-building scenes in Quest 3 or Apple Vision Pro.
• Real-time content creation: Used for web-based real-time collaboration and content creation, supporting real-time editing, relighting, and dynamic effects adjustment directly on AI-generated 3D worlds in the browser.
• Online exhibitions and e-commerce: Used for online displays in museums, art exhibitions, and real estate, enabling users to instantly browse large-scale digital scenes through progressive streaming technology without waiting for a full download.
• Web games and virtual events: Used for open-world web games and virtual concerts, enabling streaming scene loading while playing and downloading, and supporting real-time interactive rendering of ultra-large-scale virtual environments.