The Development Of 3D Reconstruction in The Field of Autonomous Driving Based on NeRF Methods

Han Wu

doi:10.54097/2a314516

Authors

Han Wu

DOI:

https://doi.org/10.54097/2a314516

Keywords:

Neural radiance field; 3D reconstruction; autonomous driving; dynamic scenes reconstruction; large-scale scene reconstruction.

Abstract

The process of 3D reconstruction is essential for the functionality of autonomous vehicles.3D reconstruction technology can take the 2D visual image information and other data collected by sensors and reconstruct a three-dimensional scene of the surrounding environment, providing autonomous vehicles with more precise environmental perception. As one of the most capable and influential 3D reconstruction methods in recent years, NeRF has become a focal point of interest and extensive investigation within the realm of autonomous driving technology.This paper concentrate on the application of the NeRF method in 3D reconstruction within the field of autonomous driving. Aiming at the NeRF with high resolution and high detail reconstruction capabilities, it first introduces the implementation principle of the original NeRF method, and then introduces various improved NeRF methods based on this approach. Subsequently, it presents the features of the improved NeRF methods as well as their shortcomings and difficulties in 3D reconstruction for autonomous driving. Finally, the future development trend of the NeRF method is discussed and predicted.

Downloads

Download data is not yet available.

References

[1] He, L., Li, L., Sun, W., Han, Z., Liu, Y., Zheng, S., Wang, J., Li, K.: 'Neural Radiance Field in Autonomous Driving: A Survey', ArXiv, 2024, abs/2404.13816

[2] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: 'NeRF', Communications of the ACM, 2020, 65, pp. 99-106

[3] Ost, J., Mannan, F., Thuerey, N., Knodt, J., Heide, F.: 'Neural Scene Graphs for Dynamic Scenes', 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 2855-2864

[4] Tancik, M., Casser, V., Yan, X., Pradhan, S., Mildenhall, B., Srinivasan, P.P., Barron, J.T., Kretzschmar, H.: 'Block-NeRF: Scalable Large Scene Neural View Synthesis', 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 8238-8248

[5] Rematas, K., Liu, A., Srinivasan, P.P., Barron, J.T., Tagliasacchi, A., Funkhouser, T.A., Ferrari, V.: 'Urban Radiance Fields', 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12922-12932

[6] Weng, C., Curless, B., Srinivasan, P.P., Barron, J.T., Kemelmacher-Shlizerman, I.: 'HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video', 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 16189-16199

[7] Jiang, Y., Yao, K., Su, Z., Shen, Z., Luo, H., Xu, L.: 'Instant-NVR: Instant Neural Volumetric Rendering for Human-object Interactions from Monocular RGBD Stream', 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 595-605

[8] Yang, J., Ivanovic, B., Litany, O., Weng, X., Kim, S.W., Li, B., Che, T., Xu, D., Fidler, S., Pavone, M., Wang, Y.: 'EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision', ArXiv, 2023, abs/2311.02077

[9] Zhang, X., Bi, S., Sunkavalli, K., Su, H., Xu, Z.: 'NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction', 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5439-5448

[10] Liu, L., Gu, J., Lin, K.Z., Chua, T., Theobalt, C.: 'Neural Sparse Voxel Fields', ArXiv, 2020

[11] Müller, T., Evans, A., Schied, C., Keller, A.: 'Instant neural graphics primitives with a multiresolution hash encoding', ACM Transactions on Graphics (TOG), 2022, 41, pp. 1-15