Multimodal Perception Technology, Fusion, and Application of Robot Dexterous Hands for Complex Tasks in Intelligent Manufacturing

Abstract

To meet the flexible production line requirements of smart manufacturing characterized by variety, small batches, and sub-millimeter precision, traditional single-mode vision solutions suffer from large pose errors and slow production changes in scenarios involving occlusion, reflection, weak textures, and flexible objects. This paper systematically reviews the technology, fusion, and application progress of multimodal perception (visual, tactile, and force) for robotic dexterous hands. First, it outlines the principles of vision and high-resolution tactile sensing, as well as six-dimensional force/torque sensing. Subsequently, it proposes a task-oriented framework, comparing the advantages and disadvantages of visual-guided tactile verification serial strategies versus end-to-end joint modeling in 6D pose estimation. It summarizes the millisecond-level closed-loop effectiveness of slip detection-grip force regulation and near-range tactile collaboration in stable grasping and dexterous operations. Through case studies of two typical production lines—high-precision assembly and flexible object manipulation—this paper identifies future breakthrough directions: low-cost tactile sensors, few-shot cross-modal alignment, production-line-level datasets, and interpretable fusion frameworks. The aim is to provide a methodology and a roadmap for the transition of dexterous hands from laboratory settings to standard batch production.