Abstract
The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools. Here, we focus on inpainting regions in a view-consistent and controllable manner. In addition to the typical NeRF inputs and masks delineating the unwanted region in each view, we require only a single inpainted view of the scene, i.e., a reference view. We use monocular depth estima- tors to back-project the inpainted view to the correct 3D positions. Then, via a novel rendering technique, a bilat- eral solver can construct view-dependent effects in non- reference views, making the inpainted region appear consis- tent from any view. For non-reference disoccluded regions, which cannot be supervised by the single reference view, we devise a method based on image inpainters to guide both the geometry and appearance. Our approach shows superior performance to NeRF inpainting baselines, with the addi- tional advantage that a user can control the output via a single inpainted image.
Toronto Intelligent Systems Lab Co-authors
Ashkan Mirzaei
PhD Student
My research interests include 3D Representation and Scene Manipulation. My current focus is to distill 2D knowledge into 3D.
