A Reflective Object Reconstruction Method Based on Neural Radiance Field

Lifen Li; Zhengqin Yu; Ronghua Zhang

doi:10.54097/3nzpbc54

Authors

Lifen Li
Zhengqin Yu
Ronghua Zhang

DOI:

https://doi.org/10.54097/3nzpbc54

Keywords:

Neural Radiance Field, Neural Implicit Surface, Multi-view Reconstruction, Computer Vision.

Abstract

We propose a neural rendering-based 3D reconstruction method for reconstructing the geometry and BRDF of reflective objects from multi-view images captured in unknown environments. Multi-view reconstruction of reflective objects is extremely challenging because specular reflections are view-dependent and therefore violate multi-view consistency, which is the cornerstone of most multi-view reconstruction methods. Recent neural rendering techniques can model the interaction between ambient light and object surfaces to adapt to view-dependent reflections, making it possible to reconstruct reflective objects from multi-view images. However, accurately modeling ambient light in neural rendering is difficult. We propose a two-step approach to solve this problem. First, by introducing view-dependent photometric losses, our method accurately reconstructs the geometry of reflective objects. Then, with the object geometry fixed, we use more accurate sampling to recover the BRDF of the ambient light and object. Experiments show that our method is able to accurately reconstruct the geometry and BRDF of reflective objects from only RGB images without knowing the ambient light and object masks.

Downloads

Download data is not yet available.

References

Hartley R, Zisserman A. Multiple view geometry in computer vision[M]. Cambridge university press, 2003.

Kazhdan M, Bolitho M, Hoppe H. Poisson surface reconstruction[C]//Proceedings of the fourth Eurographics symposium on Geometry processing. 2006, 7(4).

Cardozo N, Allmendinger R W. Spherical projections with OSXStereonet[J]. Computers & Geosciences, 2013, 51: 193-205.

Chen L, Xu J. Optimal delaunay triangulations[J]. Journal of Computational Mathematics, 2004: 299-308.

Schonberger J L, Frahm J M. Structure-from-motion revisited[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 4104-4113.

Mildenhall B, Srinivasan P P, Tancik M, et al. Nerf: Representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.

Wang P, Liu L, Liu Y, et al. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction[J]. arXiv preprint arXiv:2106.10689, 2021.

Verbin D, Hedman P, Mildenhall B, et al. Ref-nerf: Structured view-dependent appearance for neural radiance fields[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 5481-5490.

Guo Y C, Kang D, Bao L, et al. Nerfren: Neural radiance fields with reflections[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 18409-18418.

Ge W, Hu T, Zhao H, et al. Ref-neus: Ambiguity-reduced neural implicit surface learning for multi-view reconstruction with reflection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 4251-4260.

Liu Y, Wang P, Lin C, et al. Nero: Neural geometry and brdf reconstruction of reflective objects from multiview images[J]. ACM Transactions on Graphics (TOG), 2023, 42(4): 1-22.

Martin-Brualla R, Radwan N, Sajjadi M S M, et al. Nerf in the wild: Neural radiance fields for unconstrained photo collections[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 7210-7219.