Research on Improving Numerical Accuracy of Ocean Circulation Prediction Using Mixed-former Model

Min Xu; Li Ma; Yang Li

doi:10.54097/bkp3jh06

Authors

Min Xu
Li Ma
Yang Li

DOI:

https://doi.org/10.54097/bkp3jh06

Keywords:

Ocean Circulation Prediction, Transformer Model, Mixed-former, Multi-scale Structure, Two-stage Attention

Abstract

With the in-depth research on global climate change, accurately grasping ocean circulation patterns has become increasingly crucial for understanding the evolution laws of the climate system. However, traditional numerical models have limited prediction accuracy in ocean circulation prediction due to their insufficient characterization of multivariate spatiotemporal dependencies, especially in short-term prediction scenarios where error fluctuations are relatively large. This study aims to propose an ocean circulation prediction method based on the Mixed-Former model to improve the numerical accuracy of predictions. In the research process, we first constructed a Mixed-Former model that integrates multi-scale spatiotemporal information. This model fully retains temporal and dimensional information and effectively captures cross-variable dependencies through the innovative Dimension-Segment-Wise (DSW) embedding technology; it accurately captures temporal fluctuations using the Two-Stage Attention (TSA) layer; and it achieves efficient fusion of multi-scale information by adopting a Hierarchical Encoder-Decoder (HED) structure. Subsequently, we conducted comprehensive experiments on multiple sets of ocean circulation datasets covering different ocean regions and different spatiotemporal characteristics. These datasets include multi-time-scale prediction scenarios ranging from short-term (7 days) to long-term (six months). Meanwhile, to comprehensively and objectively evaluate the model performance, we selected several core evaluation indicators such as Mean Squared Error (MSE), Mean Absolute Error (MAE), and Correlation Coefficient (R).In conclusion, the Mixed-Former model demonstrates significant advantages in ocean circulation prediction, whether for predicting short-term fluctuations or long-term trends. It provides a more reliable tool for marine dynamics research and long-term climate prediction, and is expected to promote the further development of research in related fields.

Downloads

Download data is not yet available.

References

[1] Tahir Arshad, Junping Zhang, Shibwabo C Anyembe, et al. Spectral Spatial Neighborhood Attention Transformer for Hyperspectral Image Classification[J]. Canadian Journal of Remote Sensing, 2024, 50(1).

[2] Zhou Changjian, Li Zhongzheng, Song Jia, et al. TransVAE-DTA: Transformer and Variational Autoencoder Network for Drug-Target Binding Affinity Prediction[J]. Computer Methods and Programs in Biomedicine, 2023, 244: 108003.

[3] Rui Wang, Yujin Ji, Youyong Li, et al. Applications of Transformers in Computational Chemistry: Recent Progress and Prospects[J]. The Jour- nal of Physical Chemistry Letters, 2024, (): 421-434.

[4] Minyahil Tanashu, Tassew Tadiwos, Amare Kassaw, et al. Design and Analysis of Low-Cost and Efficient SEPIC Converter for Affordable So- lar Water Pump in Ethiopia[J]. International Journal of Engineering Re- search in Africa, 2024, 72: 47-69.

[5] Ali M. Jasim, Alaa Hussein Abdulaal, Baraa M. Albaker, et al. High - Gain Cubic Boost Converter Analysis with Hybrid ANN - Incremental Conductance MPPT for Solar PV Systems[J]. Mathematical Modelling of Engineering Problems, 2024, 11(12).

[6] Zhihong Cao, Kun Liang, Sheng Tang, et al. Applications of the Fusion- ScratchNet Algorithm Based on Convolutional Neural Networks and Transformer Models in the Detection of Cell Phone Screen Scratches[J]. Electronics, 2024, 14(1): 134.

[7] Wei Huo, Xiaodan Zhang, Shaojie You, et al. STGAN: Swin Trans- former - Based GAN to Achieve Remote Sensing Image Super - Reso- lution Reconstruction[J]. Applied Sciences, 2024, 15(1): 305.

[8] Zhou Changjian, Li Zhongzheng, Song Jia, et al. TransVAE - DTA: Transformer and variational autoencoder network for drug - target binding affinity prediction[J]. Computer methods and programs in biomedicine, 2023, 244: 108003.

[9] Kassim Rasheed Hameed, Ahlam Luaibi Suraiji, Abduljabbar O. Han- fesh. Comparative the effect of distribution transformer coil shape on electromagnetic forces and their distribution using the FEM[J]. Open Engineering, 2023, 13(1).

[10] Zhou Yulong, Hussain Muhammad Afaq, Chen Zhanlong. Landslide susceptibility mapping with feature fusion transformer and machine learning classifiers incorporating displacement velocity along Karako- ram highway[J]. Geocarto International, 2023, 38(1).

[11] Niu Bowen, Feng Quanlong, Yang Jianyu, et al. Solid waste mapping based on very high resolution remote sensing imagery and a novel deep learning approach[J]. Geocarto International, 2023, 38(1).

[12] Ashok B., Michael Prawin Angel. A non - isolated high step - up con- verter with TID controller for solar photovoltaic integrated with EV[J]. Analog Integrated Circuits and Signal Processing, 2023, 118(3): 505 – 521.

[13] A Madhavi, A Nagesh, A Govardhan. Efficient Course Recommenda- tion using Deep Transformer based Ensembled Attention Model[J]. EAI Endorsed Transactions on e - Learning, 2023, 9(1).

[14] Piyush Bhushan Singh, Pawan Singh, Harsh Dev, et al. Glaucoma Clas- sification using Light Vision Transformer[J]. EAI Endorsed Transac- tions on Pervasive Health and Technology, 2023, 9(1).

[15] Anil V Turukmane, Sagar Pande, Vaidehi Bedekar, et al. Reinforced Hybrid Graph Transformer for Medical Recommendations[J]. EAI En- dorsed Transactions on Pervasive Health and Technology, 2023, 9(1).

[16] Yuan Panli, Zhao Qingzhan, ZhaoXingbiao, et al. A transformer - based Siamese network and an open optical dataset for semantic change detec- tion of remote sensing images[J]. International Journal of Digital Earth, 2022, 15(1): 1506 – 1525.

[17] Yu Zaiyang, Li Shuang, Sun Linjun, et al. Multi - distribution noise quantisation: an extreme compression scheme for transformer according to parameter distribution[J]. Connection Science, 2022, 34(1): 990 – 1004.

[18] Zhu Yuan, Xia Qingyuan, Jin Wen. SRDD: a lightweight end - to - end object detection with transformer[J]. Connection Science, 2022, 34(1): 2448 – 2465.

[19] He Qiang, Nie Jingkai, Han Yu, et al. Investigation on Low Frequency Bandgap of Coupled Double Beam with Quasi - Zero Stiffness for Power Transformer Vibration Control[J]. Shock and Vibration, 2022, 2022.

[20] Ala Houam, Fares Zaamouche, Rabah Daouadi, et al. Efficiency Con- siderations of LLC Resonant Converter for Induction Heating Applica- tion[J]. European Journal of Electrical Engineering, 2022, 24(5 - 6).

[21] Chen Guzhong, Song Zhen, Qi Zhiwen. Transformer - convolutional neural network for surface charge density profile prediction: Enabling high - throughput solvent screening with COSMO - SAC[J]. Chemical Engineering Science, 2021, 246.

[22] Kalyan Katikapalli Subramanyam, Rajasekharan Ajit, Sangeetha Sivanesan. AMMU: A Survey of Transformer - based Biomedical Pre- trained Language Models[J]. Journal of Biomedical Informatics, 2021, (prepublish): 103982.