3D Point Cloud Semantic Segmentation based on Multi-scale Dense Nested Networks

Zishuo Wang; Tianxiang Lai; Yufeng Wang; Xingquan Gao

doi:10.54097/wx687k80

Authors

Zishuo Wang
Tianxiang Lai
Yufeng Wang
Xingquan Gao

DOI:

https://doi.org/10.54097/wx687k80

Keywords:

Point Cloud Semantic Segmentation, Nested Networks, Multi-scale Feature Fusion, Dense Connection

Abstract

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

Downloads

Download data is not yet available.

References

[1] Wang C, Pastore F, Goknil A, et al. Automatic generation of acceptance test cases from use case specifications: an nlp-based approach[J]. IEEE Transactions on Software Engineering, 2020, 48(2): 585-616.

[2] Kolhatkar C, Wagle K. Review of SLAM algorithms for indoor mobile robot with LIDAR and RGB-D camera technology[J]. Innovations in Electrical and Electronic Engineering: Proceedings of ICEEE 2020, 2021: 397-409.

[3] Zaman F, Wong Y P, Ng B Y. Density-based denoising of point cloud[C]//9th International Conference on Robotic, Vision, Signal Processing and Power Applications: Empowering Research and Innovation. Springer Singapore, 2017: 287-295.

[4] Mandikal P, Radhakrishnan V B. Dense 3d point cloud reconstruction using a deep pyramid network[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2019: 1052-1060.

[5] Diab A, Kashef R, Shaker A. Deep learning for LiDAR point cloud classification in remote sensing[J]. Sensors, 2022, 22(20): 7868.

[6] Zhang J, Zhao X, Chen Z, et al. A review of deep learning-based semantic segmentation for point cloud[J]. IEEE access, 2019, 7: 179118-179133.

[7] Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3d shape recognition[C]//Proceedings of the IEEE international conference on computer vision, 2015: 945-953.

[8] Boulch A, Guerry J, Le Saux B, et al. SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks[J]. Computers & Graphics, 2018, 71: 189-198.

[9] Riegler G,Ulusoy O A,Geiger A. OctNet: Learning Deep 3D Representations at High Resolutions.[J]. CoRR,2016,abs/ 1611. 05009.

[10] Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 652-660.

[11] Qi C R, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space[J]. Advances in neural information processing systems, 2017, 30.

[12] Hu Q, Yang B, Xie L, et al. Randla-net: Efficient semantic segmentation of large-scale point clouds[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020: 11108-11117.

[13] Li Y, Bu R, Sun M, et al. Pointcnn: Convolution on x-transformed points[J]. Advances in neural information processing systems, 2018, 31.

[14] Wu W, Qi Z, Fuxin L. Pointconv: Deep convolutional networks on 3d point clouds[C]//Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 2019: 9621-9630.

[15] Du J and Cai G R. 2021. Point cloud semantic segmentation method based on multi-feature fusion and residual optimization [J]. Journal of Image and Graphics,2021, (05): 1105-1116.

[16] Liu Z, Hu H, Cao Y, et al. A closer look at local aggregation operators in point cloud analysis[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII 16. Springer International Publishing, 2020: 326-342.

[17] Xu M, Ding R, Zhao H, et al. Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3173-3182.

[18] HAO W, WANG H X, WANG Y. Semantic segmentation of three-dimensional point cloud based on spatial attention and shape feature [J]. Laser and Optoelectronics Progress, 2022, 59 (8): No. 0828004.

[19] Lin Y, Chen L, Huang H, et al. Beyond farthest point sampling in point-wise analysis[J]. arXiv preprint arXiv:2107.04291, 2021.

[20] Groh F, Wieschollek P, Lensch H P A. Flex-Convolution: Million-scale point-cloud learning beyond grid-worlds[C]// Asian Conference on Computer Vision. Cham: Springer International Publishing, 2018: 105-122.

[21] Targ S, Almeida D, Lyman K. Resnet in resnet: Generalizing residual architectures[J]. arXiv preprint arXiv:1603.08029, 2016.

[22] Lee C Y, Xie S, Gallagher P, et al. Deeply-supervised nets [C]// Artificial intelligence and statistics. Pmlr, 2015: 562-570.

[23] Ke L, Chang M C, Qi H, et al. Multi-scale structure-aware network for human pose estimation[C]//Proceedings of the european conference on computer vision (ECCV), 2018: 713-728.

[24] Armeni I, Sener O, Zamir A R, et al. 3d semantic parsing of large-scale indoor spaces[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: 1534-1543.

[25] Babacan K, Chen L, Sohn G. Semantic segmentation of indoor point clouds using convolutional neural network[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 4: 101-108.

[26] Rottensteiner F, Sohn G, Gerke M, et al. Results of the ISPRS benchmark on urban object detection and 3D building reconstruction [J]. ISPRS journal of photogrammetry and remote sensing, 2014, 93: 256-271.

[27] Phan A V, Le Nguyen M, Nguyen Y L H, et al. Dgcnn: A convolutional neural network over large-scale labeled graphs [J]. Neural Networks, 2018, 108: 533-543.

[28] Deng S, Dong Q. GA-NET: Global attention network for point cloud semantic segmentation[J]. IEEE Signal Processing Letters, 2021, 28: 1300-1304.

[29] Zhao H, Jiang L, Fu C W, et al. Pointweb: Enhancing local neighborhood features for point cloud processing[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019: 5565-5573.

[30] Shuai H, Xu X, Liu Q. Backward attentive fusing network with local aggregation classifier for 3D point cloud semantic segmentation [J]. IEEE Transactions on Image Processing, 2021, 30: 4973-4984.

[31] Fan S, Dong Q, Zhu F, et al. SCF-Net: Learning spatial contextual features for large-scale point cloud segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 14504-14513.