Research And Future Application Analysis of Multimodal Fusion
DOI:
https://doi.org/10.54097/sx342m55Keywords:
Multimodal fusion, machine learning, early fusion, late fusion, hybrid fusion sentiment analysis.Abstract
This manuscript delves into the origin, progression, and future of multimodal fusion by conducting a comprehensive review of seminal research at various phases of its ontogeny. It encompasses three foundational methodologies of multimodal integration: Early Fusion, late Fusion, and Hybrid Fusion. Moreover, the article presents three novel methodologies for integration. It culminates in a discourse on potential subsequent applications, obstacles, and progress within the domain. solve the fusion problem of heterogeneous neural networks, and improve the correlation and consistency between modes in the aspects of cross-modal representation learning, multi-modal sentiment analysis, multi-modal intelligent interaction, so as to achieve better human-computer interaction experience.
Downloads
References
[1] Fan Weiquan, He Zhiwei, Xing Xiaofen, Cai Bolun, Lu. Weirui Multi-modality Depression Detection via Multi-scale Temporal Dilated CNNs Session: Detecting Depression with AI Sub-challenge, 2019.
[2] Yishan Chen, Zhiyang Jia, Kaoru Hirota, Yaping Dai. P A Multimodal Emotion Perception Model based on Context-Aware Decision-Level Fusion, roceedings of the 41st Chinese Control Conference, 2022.
[3] J. Ye, W. Zheng, and L. Yang, et al, Multimodal emotion recognition based on deep neural network, Journal of Southeast University (English Edition), 2017, 33 (4): 444 - 447.
[4] F. Xu and Z. Wang, Emotion Recognition Research Based on Integration of Facial Expression and Voice, 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 2018: 1 - 6.
[5] Q. Xu, B. Sun, and J. He, et al, Multimodal Facial Expression Recognition Based on Dempster-Shafer Theory Fusion Strategy, 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia), 2018: 1 - 5.
[6] F. Karim, S. Majumdar, H. Darabi and S. Chen, LSTM Fully Convolutional Networks for Time Series Classification, in IEEE Access, 2018, 6: 1662 - 1669.
[7] Guang Yang Jiaxin Li Deguo Yang, Guojun Wang 2023 5th International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI) Multimodal Emotion Recognition Based on Hybrid Fusion, 2023.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Highlights in Science, Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
