Building Damage Degree Recognition Based on Temporal Attention Features

Zhenzhao Jiang

doi:10.54097/73nxxv59

Authors

Zhenzhao Jiang

DOI:

https://doi.org/10.54097/73nxxv59

Keywords:

Structure; damage assessment; semantic change-detection.

Abstract

Natural disasters pose significant harm to society. As an important place for social activities and economic development, the degree of damage to building areas is directly related to disaster loss assessment and emergency rescue. Remote sensing image data, characterized by its wide coverage and multi-temporal features, provides important data support for post-disaster loss assessment. However, imaging differences caused by factors such as shooting time, imaging angle, and different sensors can interfere with the extraction of damage features and loss assessment. This paper proposes a Dual-Exchange-Attention U-Net (DERU-Net) model, which transforms the identification of building damage levels into intra-class semantic change detection. The DFMA feature attention fusion module is introduced to enhance the ability of dual-temporal feature extraction and achieve end-to-end assessment of building damage. The proposed method is comprehensively evaluated and tested on the xBD dataset. Experimental results show that compared with other methods, the DERU-Net proposed in this paper exhibits better stability and evaluation accuracy in assessing the degree of building damage.

Downloads

Download data is not yet available.

References

ENGEL C B, JONES S D, REINKE K. A seasonal-window ensemble-based thresholding technique used to detect active fires in geostationary remotely sensed data [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(6): 4947-56.

PIERDICCA N, ANNIBALLE R, NOTO F, et al. Triple collocation to assess classification accuracy without a ground truth in case of earthquake damage assessment [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(1): 485-96.

YAMAGUCHI T, MIZUTANI T, TARUMI M, et al. Sensitive damage detection of reinforced concrete bridge slab by “time-variant deconvolution” of SHF-band radar signal [J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 57(3): 1478-88.

RITWIK G. xbd: A dataset for assessing building damage from satellite imagery [J]. arXiv preprint, 2019.

GUPTA R, SHAH M. Rescuenet: Joint building segmentation and damage assessment from satellite imagery; proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), F, 2021 [C]. IEEE.

WEBER E, KANé H. Building disaster damage assessment in satellite imagery with multi-temporal fusion [J]. arXiv preprint arXiv:200405525, 2020.

SU J, BAI Y, WANG X, et al. Technical solution discussion for key challenges of operational convolutional neural network-based building-damage assessment from satellite imagery: Perspective from benchmark xBD dataset [J]. Remote Sensing, 2020, 12(22): 3808.

CHEN S A, ESCAY A, HABERLAND C, et al. Benchmark dataset for automatic damaged building detection from post-hurricane remotely sensed imagery [J]. arXiv preprint arXiv:181205581, 2018.

GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD: A dataset for assessing building damage from satellite imagery; proceedings of the Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, F, 2019 [C].

FOULSER-PIGGOTT R, SPENCE R, SAITO K, et al. The use of remote sensing for post-earthquake damage assessment: lessons from recent events, and future prospects; proceedings of the Proceedings of the Fifthteenth World Conference on Earthquake Engineering, F, 2012 [C].

DONG L, SHAN J. A comprehensive review of earthquake-induced building damage detection with remote sensing techniques [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 84: 85-99.

CI T, LIU Z, WANG Y. Assessment of the degree of building damage caused by disaster using convolutional neural networks in combination with ordinal regression [J]. Remote Sensing, 2019, 11(23): 2858.

SUMER E, TURKER M. Building damage detection from post-earthquake aerial imagery using building grey-value and gradient orientation analyses; proceedings of the Proceedings of 2nd International Conference on Recent Advances in Space Technologies, 2005 RAST 2005, F, 2005 [C]. IEEE.

YE X, LIU M, WANG J, et al. Building-based damage detection from postquake image using multiple-feature analysis [J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(4): 499-503.

RUDNER T G, RUßWURM M, FIL J, et al. Rapid Computer Vision-Aided Disaster Response via Fusion of Multiresolution, Multisensor, and Multitemporal Satellite Imagery; proceedings of the Proceedings of the First Workshop on AI for Social Good Neural Information Processing Systems (NIPS-2018), Montreal, QC, Canada, F, 2018 [C].

ZHU Z. Change detection using landsat time series: A review of frequencies, preprocessing, algorithms, and applications [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 370-84.

TEWKESBURY A P, COMBER A J, TATE N J, et al. A critical synthesis of remotely sensed optical image change detection techniques [J]. Remote Sensing of Environment, 2015, 160: 1-14.

BLASCHKE T, KELLY M, MERSCHDORF H. Object-based image analysis: Evolution, history, state of the art, and future vision [M]. 2015.

ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks; proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, F, 2015 [C].

XIAO H, PENG Y, TAN H, et al. Dynamic cross fusion network for building-based damage assessment; proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), F, 2021 [C]. IEEE.

FU X, KOUYAMA T, YANG H, et al. Toward Faster and Accurate Post-Disaster Damage Assessment: Development of End-to-End Building Damage Detection Framework with Super-Resolution Architecture; proceedings of the IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, F, 2022 [C]. IEEE.

FILIPE S, ALEXANDRE L A. From the human visual system to the computational models of visual attention: a survey [J]. Artificial Intelligence Review, 2013, 39(1): 1-47.

HU J, SHEN L, SUN G. Squeeze-and-excitation networks; proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, F, 2018 [C].

GUO J, MA X, SANSOM A, et al. Spanet: Spatial pyramid attention network for enhanced image recognition; proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), F, 2020 [C]. IEEE.

WOO S, PARK J, LEE J-Y, et al. Cbam: Convolutional block attention module; proceedings of the Proceedings of the European conference on computer vision (ECCV), F, 2018 [C].

SHEN Y, ZHU S, YANG T, et al. Bdanet: Multiscale convolutional neural network with cross-directional attention for building damage assessment from satellite images [J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-14.

HAO H, BAIREDDY S, BARTUSIAK E R, et al. An attention-based system for damage assessment using satellite imagery; proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, F, 2021 [C]. IEEE.

WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks; proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, F, 2018 [C].

ZHAO S, ZHANG X, XIAO P, et al. Exchanging Dual-Encoder–Decoder: A New Strategy for Change Detection With Semantic Guidance and Spatial Localization [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-16.

RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation; proceedings of the Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, F, 2015 [C]. Springer.

LIU L, CHENG J, QUAN Q, et al. A survey on U-shaped networks in medical image segmentations [J]. Neurocomputing, 2020, 409: 244-58.

[XU G, LIAO W, ZHANG X, et al. Haar wavelet downsampling: A simple but effective downsampling module for semantic segmentation [J]. Pattern Recognition, 2023, 143: 109819.

DAUDT R C, LE SAUX B, BOULCH A. Fully convolutional siamese networks for change detection; proceedings of the 2018 25th IEEE international conference on image processing (ICIP), F, 2018 [C]. IEEE.

FANG S, LI K, SHAO J, et al. SNUNet-CD: A densely connected Siamese network for change detection of VHR images [J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19: 1-5.

CHEN P, ZHANG B, HONG D, et al. FCCDN: Feature constraint network for VHR image change detection [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 187: 101-19.

ZHANG C, YUE P, TAPETE D, et al. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 183-200.

ZHANG L, HU X, ZHANG M, et al. Object-level change detection with a dual correlation attention-guided detector [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177: 147-60.

BUSLAEV A, IGLOVIKOV V I, KHVEDCHENYA E, et al. Albumentations: fast and flexible image augmentations [J]. Information, 2020, 11(2): 125.

YUN S, HAN D, OH S J, et al. Cutmix: Regularization strategy to train strong classifiers with localizable features; proceedings of the Proceedings of the IEEE/CVF international conference on computer vision, F, 2019 [C].

PASZKE A, GROSS S, MASSA F, et al. Pytorch: An imperative style, high-performance deep learning library [J]. Advances in neural information processing systems, 2019, 32.

KINGMA D P, BA J. Adam: A method for stochastic optimization [J]. arXiv preprint arXiv:14126980, 2014.