UniWorld-View - An Open-Source World Model by Tuzhan Intelligence Jointly with Peking University and Others

What is UniWorld-View

UniWorld-View is an open-source world model launched by Tuzhan Intelligence in collaboration with Peking University and Pengcheng Laboratory, and it topped the WorldScore world model evaluation leaderboard by Fei-Fei Li's team. Based on a single image or a video, the model can generate new viewpoint frames along a specified camera trajectory, achieving a unified architecture for single-image 3D generation and video 4D generation. It has completed adaptation to domestic Ascend computing power, and both the code and weights are open-sourced.

Main Features of UniWorld-View

• Single-Image Novel View Synthesis: Input a single image and generate multi-view frames along a specified camera path, supporting push, pull, pan, tilt, and 360° orbit.
• Dynamic Video Novel View Synthesis: Input a monocular video, infer unshot space and generate novel-view dynamic videos while maintaining temporal consistency.
• Unified Generation Framework: The same set of models and code supports novel view generation tasks for both static images and dynamic videos.
• Camera Trajectory Control: Supports precise camera pose control and enables large-baseline viewpoint switching.

Technical Principles of UniWorld-View

• Occlusion-Aware Point Cloud Rendering: To address foreground-background tearing and backface light leakage in point cloud rendering, it proposes dual reprojection and normal filtering mechanisms.
• Double-Reprojection: After projecting the source image to the target viewpoint, it is projected back along the original path. Pixels that cannot return are occluded regions, which are accumulated frame by frame into an occlusion mask to prevent erroneous textures from misleading the generative model.
• Normal-Filtering: Uses normal information to remove backface point clouds, preventing front-face pixels from penetrating and leaking when the camera moves behind an object.
• Fusion of Geometry and Generation: First uses a geometric model to construct a 3D point cloud and render a target-viewpoint condition image, then feeds it into a video diffusion model for generation, balancing camera control precision and visual quality.

Follow WeChat and reply “open source” to join theAI Open-Source Project Communication Group

How to Use UniWorld-View

• Environment Preparation: Clone the GitHub repository, configure the Python environment, and install dependencies.
• Model Download: Download the open-source weights from Hugging Face or GitHub Release.
• Data Input: Prepare a single image or a video as the source material, and define the target camera trajectory.
• Run Inference: Execute the inference script, and the model will automatically generate new frames or videos from the specified viewpoint.
• Ascend Deployment: If domestic computing power is needed, complete Ascend NPU adaptation and inference acceleration according to the official documentation.

Project Address of UniWorld-View

• GitHub Repository：https://github.com/PKU-YuanGroup/UniWorld-View
• HuggingFace Model Library：https://huggingface.co/Drexubery/UniView

Comparison of UniWorld-View with Similar Competing Products

Application scenarios of UniWorld-View

• Film and advertising production: Rapidly generate multi-camera previews of scenes, reducing the cost of live-action shooting and 3D reconstruction.
• Virtual reality and gaming: Generate interactive 360° surround scenes based on a single concept image or video.
• Robotics and autonomous driving simulation: Construct novel-view data of the real world for training visual perception and path planning models.
• Cultural heritage digitization: After collecting a single image or single video of a cultural relic or ancient building, generate a digital display with surround views.
• E-commerce and real estate: Through a single product image or room video, generate multi-angle display content to enhance user immersion.