SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI images一键直达

一键直达

Farm3D is a single-view 3D reconstruction and controllable 3D synthesis method proposed by a team at the University of Oxford, with the paper published at 3DV 2024. This method does not rely on real animal images for training; instead, it obtains "free" virtual supervision signals from image generators based on 2D diffusion models (such as Stable Diffusion) to train a monocular articulated object reconstruction network from scratch. Users only need to input a real or generated animal image, and the model can generate controllable 3D assets within seconds, supporting operations such as relighting, texture swapping, and animation. The research also released the Animodel 3D articulated animal dataset and benchmark, containing professionally produced textured 3D meshes and real articulated poses of horses, cows, sheep, and others.

Open website

About this tool

Purpose

Farm3D is aimed at single-view 3D reconstruction and controllable 3D synthesis of articulated objects. Its core goal is to recover a 3D shape with articulated structure from a single input image (real or generated by a diffusion model), and on this basis achieve editable and drivable 3D asset generation. The paper authors are Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi (University of Oxford), published at 3DV 2024.

Main capabilities

  • Monocular 3D reconstruction: reconstruct the 3D shape of articulated objects from a single image, capable of restoring details such as legs and ears, while not using any real images for training.
  • Controllable 3D synthesis: the generated 3D assets support adjusting lighting (Relighting), swapping textures between models of the same category (Texture swapping), and animating the shape (Animation).
  • Controllable synthetic content: the input can be either a real image or an image synthesized by Stable Diffusion.

Usage

According to the official text, the workflow of this method is as follows:

  1. Prompt Stable Diffusion to generate virtual viewpoint images of a certain object category, and use these images to train a monocular articulated object reconstruction model.
  2. The model decomposes the input image into articulated shape, appearance (albedo as well as diffuse and ambient intensity), viewpoint, and lighting direction.
  3. During training, synthesized instance views are also sampled and handed to Stable Diffusion to "judge," in order to guide learning. The official page provides entrances to the paper, BibTeX, and Code and Benchmark, but the materials do not give specific installation commands or running steps, so they are not provided for now.

Applicable scenarios

  • Research on single-image 3D reconstruction of articulated animals (such as horses, cows, and sheep).
  • 3D content creation and research experiments that require quickly obtaining controllable 3D assets from a single image.
  • Evaluation of single-view articulated object reconstruction methods, relying on the Animodel dataset and benchmark released by the team.

Dataset and evaluation

The research proposes a new 3D Articulated Animals Dataset, used to directly evaluate the quality of single-view 3D reconstruction of articulated animals. This dataset contains textured 3D meshes of articulated animals such as horses, cows, and sheep produced by professional 3D artists, and is equipped with real articulated poses.

Limitations

Official materialsdo not explain the commercial licensing, charging model, deployment platform, and performance metrics of this method, so they are not provided for now. Its capability description is concentrated on the articulated animal category and single-view reconstruction scenarios; the performance on other categories and broader scenarios is not explained in the materials, so it is not provided for now.

Related tools

Browse category