Expression Recognition Based on Multi-level Multi-model Fusion Deep Convolutional Neural Network

Abstract

In the research of Facial Expression Recognition (FER), training networks using deep learning methods often relies on data sets with accurate label and balanced number of each classification. However, when training on low-quality data sets, the performance of traditional models is insufficient. In order to solve the problem, the paper proposes an expression recognition method based on Multi-level Multi-model Fusion Deep Convolutional Neural Network. The Deep Convolutional Neural Network is composed of three models, ResNet-50, ResNet-101 and InceptionNet-V3, which have similar performance on ImageNet. We use the weighted average fusion algorithm as our fusion method. According to the accuracy of the three models, weighted fusion is carried out, and the model with high accuracy is given higher weight. It increases the width of the network and the number of parameters. At the same time, it can solve the problem of accuracy degradation when the number of layers in the deep network increases. We choose FER2013, a widely recognized low-quality data set, for the experiment. The experimental results based on FER2013 data set show that the facial expression recognition accuracy of this method reach 71.78%, which is significantly better than ResNet-50, ResNet-101 and InceptionNet-V3. Compared with other models, it also verifies the effectiveness of the proposed model in identifying low-quality data sets.