Lightweight Dynamic Gesture Recognition based on shufflenetv2-Mamba Hybrid Architecture

Abstract

Dynamic gesture recognition has important application value in human-computer interaction of mobile terminal, but the existing methods generally face the problems of high computational complexity and insufficient time sequence modeling ability. Therefore, this paper proposes a lightweight dynamic gesture recognition model based on shufflenetv2 Mamba (Shuma) hybrid architecture. In this model, Mamba's state space sequence modeling module is embedded into the shufflenetv2 backbone network to achieve efficient spatio-temporal feature fusion. First, part of the convolution operation is replaced in the downsampling bottleneck layer of shufflenetv2, and Mamba's linear complexity is used to capture the long-range dependence between video frames; Secondly, a multi-scale feature dynamic fusion mechanism is designed, which combines channel shuffle and cross layer feature stitching to enhance the collaborative representation ability of local details and global motion patterns of continuous gestures. In order to further optimize the deployment efficiency, layered quantization and structured pruning technology are introduced to compress the model parameters to 2.1MB. Experiments on a specific dynamic gesture data set including first person and home monitoring show that the accuracy of gesture classification is 89.7%, which reduces the computational overhead by about 43.6% compared with the traditional 3d-cnn and cnn-lstm models. This study provides an efficient solution for real-time dynamic gesture interaction in resource constrained scenes, and verifies the effectiveness of the fusion of lightweight convolution and sequential state space model.