3DTopia

3DTopia is a two-stage text-to-3D generation model. In the first stage, a diffusion model is used to rapidly generate candidate 3D assets, and in the second stage, assets selected from the first stage are refined. The model was released on January 18, 2024.

Purpose: For text-to-3D generation, which can be completed within 5 minutes.

Main capabilities:

• First stage: Use a diffusion model to rapidly generate candidate 3D assets. Supports controlling sampling via command-line parameters (e.g., --sampler can be set to ddim for DDIM sampling; by default, 1000-step DDPM sampling is used), sampling steps (--steps is only effective for DDIM), number of samples (--samples), input text (--text), random seed (--seed), etc. By default, rendering multi-view videos and marching cubes are enabled, and can be suppressed via --no_video and --no_mcubes. The 3D voxel resolution of marching cubes can be controlled via --mcubes_res, and the rendered video resolution can be controlled via --render_res. Results are located under the results folder.
• Second stage: Refine the assets selected in the first stage, divided into two steps. In the example, the refined mesh can be found at results/default/stage2/a_robot_0_0_if2.glb. For detailed usage, please refer to threefiner.

Usage:

1. Install the environment: It is recommended to use Anaconda to manage the environment.
2. Install the second-stage refiner: Please refer to threefiner to install the second-stage mesh refiner. Installing both environments together with Pytorch 1.12.0 and CUDA 11.3 has been tested.
3. Download checkpoints (optional): gradio_demo.py and sample_stage1.py have implemented automatic checkpoint downloading. If manual download is needed, you can download the checkpoint 3dtopia_diffusion_state_dict.ckpt or model.safetensors from huggingface.
4. Run the first stage: Use the command line to run sample_stage1.py, with the specific parameters as described above.
5. Run the second stage: Refer to threefiner for the two-step refinement.

Applicable scenarios: text-to-3D generation, rapid generation of 3D assets.

Limitations:

• If you encounter the error ImportError: /lib64/libc.so.6: version 'GLIBC_2.25' not found in the second stage, you can try installing a lower version of pymeshlab: pip install pymeshlab==0.2.
• Other limitations are not provided for now.

Other information:

• 2024/03/10: Objaverse captions have been released.
• 2024/03/04: The technical report has been released.
• 2024/01/18: The text-to-3D model 3DTopia was released.

Acknowledgements: Thanks to the community for building and open-sourcing the foundation of this work, with special thanks to the code of EG3D and Stable Diffusion, as well as the dataset provided by Objaverse.