The Development and Application of Multimodal Large Models

Abstract

This paper aims to systematically review recent research on mainstream multimodal large-scale model structures, training strategies, multimodal fusion architectures, and practical applications. It focuses on the current state and future trends of multimodal large language models (MLLMs). By analyzing the structures of typical models such as CLIP, BLIP-2, PaLM-E, and GPT-4V, this paper summarizes their practical applications, common cross-modal alignment methods in multimodal large models, pre-training strategies, and standard task evaluation methods used during the modeling process. At the application level, we conducted a detailed study of multi-modal large language models (MLLMs) regarding typical use cases and performance metrics in key areas such as image-based content creation, cross-modal retrieval, visual question-answering, and multi-modal dialogue. The analysis found that MLLMs still encounter significant challenges in addressing modal hallucinations, maintaining semantic consistency, improving reasoning capabilities, and lowering training costs. This paper also summarizes the current research issues and emphasizes that future multimodal models should aim for greater generalization, modularity, controllability, and low-resource adaptability. Finally, building on existing research, the paper suggests several promising directions for further exploration, including multimodal context learning (M-ICL), visual-language chain reasoning, cross-domain knowledge transfer, and the miniaturization and deployment optimization of multimodal large models.