Review of Real-Time Monitoring and Early Warning Technology for Natural Disasters Based on Multimodal Information Fusion

Dafeng Gong; Linggui Meng; Wanle Chi; Aichun Lin; Lili Shi

doi:10.54097/tadkgn76

Authors

Dafeng Gong
Linggui Meng
Wanle Chi
Aichun Lin
Lili Shi

DOI:

https://doi.org/10.54097/tadkgn76

Keywords:

Multimodal, Information Fusion, Natural Disasters, Real-time Detection, Intelligent Early Warning

Abstract

Multimodal information fusion technology shows great potential in real-time intelligent monitoring and early warning of natural disasters. By integrating remote sensing images, ground sensor data, social media text, UAV video and other heterogeneous data sources, the technology significantly improves the accuracy of disaster monitoring and timeliness of early warning. It shows that the global economic losses caused by natural disasters from 2015 to 2024 exceeded US $3 trillion, affecting billions of people, highlighting the social and scientific significance of an efficient early warning system. This paper mainly summarizes the application progress of multimodal information fusion in earthquake, flood, and typhoon monitoring in the past decade. However, the field still faces challenges, including data heterogeneity, computational complexity, insufficient model generalization ability and cross regional collaboration difficulties. For these reasons, researchers have proposed a number of solutions, such as cross modal alignment technology, generation of confrontation network to fill the missing data, transfer learning to improve the generalization ability of the model, federal learning to ensure data privacy. Future research directions will focus on adaptive fusion algorithm, application of generative AI technology, construction of global collaboration network and promotion of low-cost technology. Facing the challenges in the future, we need interdisciplinary cooperation and technological innovation to jointly overcome the existing problems, reduce the losses caused by natural disasters, and protect the safety of human life and property.

Downloads

Download data is not yet available.

References

[1] UNDRR. Human Cost of Disasters: An Overview of the Last 20 Years. Reduction, United Nations Off Disaster Risk. Published online 2020.

[2] Sghaier M. O., Foucher S. LT. Multimodal Approach for Flood Monitoring from Time-Series Satellite Images Combining Attribute Filters and Kohonen Map. Int Geosci Remote Sens Symp. Published online 2019. doi:10.1109/ igarss. 2019. 8899289.

[3] Goodfellow, I., Bengio, Y., Courville A. Deep Learning. MIT Press. Published online 2016.

[4] Vaswani A., Shazeer N. PN. Attention Is All You Need. Adv Neural Inf Process Syst. 2017; 30:5998-6008.

[5] Allen R M KH. The Potential for Earthquake Early Warning in Southern California. Science (80- ). 2003;300(5620):786-789. doi:10.1126/science.1080912.

[6] Zhou Y., Guo S. LW. Aerospace remote sensing technology and major natural disaster management. Cities Disaster Reduct. 2018; 6:5. doi:10.3969/j.issn.1671-0495.2018.06.017.

[7] Hall D L LJ. An introduction to multisensor data fusion. Proc IEEE. 1997;85(1):6-23. doi:10.1109/5.554205.

[8] Qin W., Tang J. LC. A typhoon trajectory prediction model based on multimodal and multitask learning. Appl Soft Comput. 2022;122. doi: 10.1016/j.asoc.2022.108804.

[9] Ahuja S., Michael M. SM. Natural disaster detection in social media and satellite imagery. ITM Web Conf. Published online 2022. doi:10.1051/itmconf/20224403010.

[10] Goodchild, M. F., Glennon JA. Crowdsourcing geographic information for disaster response: a research frontier. Int J Digit Earth. 2010;3(3):231-241. doi:10.1080/ 1753894100375 9255.

[11] Bharambe U., Chaudhari S. PK. A Federated Learning Approach to Multimodal Data Privacy for Rapid Disaster Analysis. IGARSS 2024 - 2024 IEEE Int Geosci Remote Sens Symp. Published online 2024. doi:10.1109/ IGARSS 53475. 2024. 10641421.

[12] Chen Y., Zhang H. EDW. Real-Time Earthquake Location Based on the Kalman Filter Formulation. Geophys Res Lett. 2020; 47. doi:10.1029/2019GL086240.

[13] Wu Z., Pan S. CF. A Comprehensive Survey on Graph Neural Networks. IEEE Trans Neural Networks Learn Syst. 2021;32 (1): 4-24. doi:10.1109/TNNLS.2020.2978386.

[14] Leyva-Mayorga I., Martinez-Gost M. MM. Satellite Edge Computing for Real-Time and Very-High Resolution Earth Observation. Commun IEEE Trans. 2023;71(10):15. doi:10. 1109/ TCOMM.2023.3296584.

[15] Bock Y., Melgar D. CBW. Real-Time Strong-Motion Broadband Displacements from Collocated GPS and Accelerometers. Science (80- ). 2011;336(6082):707-710. doi:10. 1126/science.1218796.

[16] Liu J., Hao G., Tao H., Xu Y., Wang H., Jiang X. CQ. Anomaly Data Identification Method for Geological Disaster Monitoring Based on Generate Adversarial Network. Geol J China Univ. 2025; 31(02):174-184.

[17] Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee AYN. Multimodal Deep Learning. Proc 28th Int Conf Mach Learn. Published online 2011:689-696.

[18] Han S., Mao H. DWJ. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. Fiber. 2015;56(4):3-7. doi:10.48550/ arXiv. 1510. 00149.

[19] Hinton G., Vinyals O. DJ. Distilling the Knowledge in a Neural Network. Comput Sci. 2015;14(7):38-39. doi:10.4140/ TCP.n. 2015. 249.

[20] Mousavi S., Ellsworth W. ZW. Earthquake transformer—an attentive deep-learning model for simultaneous earthquake detection and phase picking. Nat Commun. Published online 2020. doi:10.1038/s41467-020-17591-w.