Abstract

Single image-to-3D generation is crucial for the creation of controllable 3D assets. As it is an underconstrained problem, we capitalize on the geometric priors from novel view (3D) generation diffusion model, and appearance priors from image (2D) generation methods to guide optimization. We observe that due to the gap between the training data of 2D and 3D diffusion models, their generated views exhibit significant appearance distinguish. For example, 2D models tend to produce more details, while 3D models achieve consistent but over-smooth results across views. Therefore, we optimize a set of 3D Gaussians by using 3D priors in the spatial domain to ensure geometric consistency, while exploiting 2D priors in the frequency domain by Fourier transform for higher visual quality. This 2D-3D hybrid Fourier Score Distillation objective function is dubbed as hy-FSD, which can be applied to existing 3D generation baselines and bring noteworthy performance gains. Based on it, we further develop an image-to-3D generation pipeline to create high-quality 3D objects within one minute, called Fourier123. Through extensive experiments, Fourier123 demonstrates efficient generation capability with fast training speed and visual-friendly appearance.

Image-to-3D

Comparison

Exported Meshes

@article{yang2024fourier123,
  title={Fourier123: One Image to High-Quality 3D Object Generation with Hybrid Fourier Score Distillation},
  author={Shuzhou Yang and Yu Wang and Haijie Li and Jiarui Meng and Xiandong Meng and Jian Zhang},
  journal={arXiv preprint arXiv:2405.20669},
  year={2024}
}