TripoSF - VAST AI's New-Generation Open-Source 3D Foundation Model

What Is TripoSF

TripoSF is a new-generation 3D foundation model launched by VAST, breaking through the bottlenecks of traditional3D modelingin detail, complex structures, and scalability. It adopts the SparseFlex representation method combined with a sparse voxel structure, storing and computing voxel information only in regions near the object surface, greatly reducing memory usage and supporting high-resolution training and inference. TripoSF introduces a 'frustum-aware partitioned voxel training' strategy to further reduce training overhead. Experiments show that TripoSF performs excellently on multiple benchmarks, with Chamfer Distance reduced by about 82% and F-score improved by about 88%.

TripoSF's Main Features

• Detail Capture Capability: Traditional 3D modeling methods often struggle with detail capture, while TripoSF can capture fine surface details and microstructures. On multiple standard benchmarks, TripoSF achieves about an 82% reduction in Chamfer Distance and about an 88% improvement in F-score.
• Topology Support: TripoSF natively supports arbitrary topology and can naturally represent open surfaces and internal structures. This gives TripoSF a clear advantage when handling complex structures such as cloth and leaves.
• Computational Resource Requirements: TripoSF greatly reduces memory usage through its sparse voxel structure. This makes TripoSF more efficient in high-resolution modeling and reduces the demand for computational resources.
• Real-Time Rendering Capability: TripoSF's frustum-aware training strategy enables it to achieve greater adaptability in dynamic and complex environments. TripoSF can use rendering loss for end-to-end training, avoiding detail degradation caused by data conversion (such as densification).
• High-Resolution Modeling: TripoSF can perform training and inference at a high resolution of 1024³, enabling TripoSF to generate more refined and realistic 3D models.

TripoSF's Technical Principles

• SparseFlex Representation Method: The core of TripoSF is the SparseFlex representation method, which draws on the advantages of NVIDIA Flexicubes and introduces a sparse voxel structure. Unlike traditional dense grids, the sparse voxel structure stores and computes voxel data only in regions near the object surface, greatly reducing memory usage. This enables TripoSF to perform training and inference at a high resolution of 1024³ while natively supporting arbitrary topological structures.
• Frustum-Aware Partitioned Voxel Training Strategy: The strategy draws on the idea of frustum culling in real-time rendering. In each training iteration, only the SparseFlex voxels located within the camera frustum are activated and processed. This targeted activation significantly reduces training overhead and makes efficient training at high resolution possible.
• TripoSF Variational Autoencoder (VAE): Based on the SparseFlex representation and efficient training strategy, VAST built the TripoSF VAE, forming a complete and efficient processing pipeline. From input, encoding, and decoding to output, the TripoSF VAE has become an important foundation for a major step forward in the TripoSF reconstruction and generation experience.

TripoSF's Project Address

• Project Website：https://xianglonghe.github.io/TripoSF/
• Github Repository：https://github.com/VAST-AI-Research/TripoSF
• HuggingFace Model Library：https://huggingface.co/VAST-AI/TripoSF
• arXiv Technical Paper：https://arxiv.org/pdf/2503.21732

TripoSF's Benchmark Test Results

• Chamfer Distance (CD) reduced by about 82%: Chamfer Distance is one of the metrics for measuring the quality of 3D model reconstruction, calculating the distance between points on the model surface and points on the true model surface. TripoSF's significant reduction in this metric indicates its superiority in capturing model details.
• F-score improved by about 88%: F-score is another metric for measuring the quality of 3D model reconstruction, comprehensively considering the model's precision and recall. TripoSF's significant improvement on this metric indicates that while maintaining model details, it can also capture the overall structure of the model well.

Comparison of TripoSF's results

Application scenarios of TripoSF

• Visual effects (VFX): TripoSF can generate high-resolution, detail-rich 3D models, making it suitable for visual effects production in fields such as film and games.
• Game development: In game development, TripoSF can be used to generate high-quality 3D game assets, such as characters, environments, and props.
• Embodied intelligence: TripoSF has broad application prospects in the field of embodied intelligence and can be used for robot simulation and interaction.
• Product design: In the field of product design, TripoSF can be used for rapid prototyping and design validation. Designers can use TripoSF to generate high-resolution 3D models for detailed design evaluation and modification.