NeRF Papers for Mon Oct 2 2023

Use the forward/backward buttons at the top left to move between days. Use the home button to jump to latest day. Use the arrow keys on the keyboard or in the bottom right to move between categories and papers. ESC enters an overview mode. On mobile touch inputs with swipes are enabled.

Fundamentals

These papers address more fundamental problems of view-synthesis with NeRF methods.

Sketch2CADScript: 3D Scene Reconstruction from 2D Sketch using Visual Transformer and Rhino Grasshopper

Hong-Bin Yang

Figure 1. An example of the uneven surface created by the 3D reconstruction method that is based on template-mesh deformation (sketch2model).

The paper introduces a 3D reconstruction method that uses a visual transformer to predict a scene descriptor from a wire-frame image. The predicted parameters can be used to reconstruct the 3D scene using modeling software. The method performs well on simple scenes but faces challenges with more complex scenes.

Priors and Generative

Priors can either aid in the reconstruction or can be used in a generative manner. For example, in the reconstruction, priors either increase the quality of neural view synthesis or enable reconstructions from sparse image collections.

HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field

Xiaochen Zhao, Lizhen Wang, Jingxiang Sun, Hongwen Zhang, Jinli Suo, Yebin Liu

Fig. 1. Our method is able to synthesize high-resolution, photo-realistic and view-consistent head images, achieving fine-grained control over head poses and facial expressions.

The Facial Model Conditioned Neural Radiance Field (NeRF) combines the expressiveness of NeRF and the prior information from a parametric template to model an animatable 3D human head avatar. By incorporating a synthetic-renderings-based condition method, the prior information is fused into the implicit field without compromising its flexibility. The proposed method achieves high-resolution, realistic, and view-consistent synthesis of dynamic head appearance, outperforming previous methods in 3D head avatar animation.

Other

Several works with excellent results in various fields.

Preface: A Data-driven Volumetric Prior for Few-shot Ultra High-resolution Face Synthesis

Marcel C. Bühler, Kripasindhu Sarkar, Tanmay Shah, Gengyan Li, Daoye Wang, Leonhard Helminger, Sergio Orts-Escolano, Dmitry Lagun, Otmar Hilliges, Thabo Beeler, Abhimitra Meka

Figure 1. We propose a method for synthesising novel views of faces at ultra high-resolution from very sparse inputs. This figure shows novel view renderings at 4K resolution reconstructed from only three views of the target identity.

The proposed prior model allows for the synthesis of high-resolution novel views of human faces with minimal input data. By training on a dataset of low-resolution multi-view images and incorporating a sparse landmark-based 3D alignment, the model can learn a smooth latent space of geometry and appearance. This enables the generation of high-quality volumetric representations from just a few casually captured images.