Privacy-Preserving Facial Emotion Classification: A Comprehensive Investigation of Federated Learning Approaches

Chun Chen

doi:10.54097/zryzda22

Authors

Chun Chen

DOI:

https://doi.org/10.54097/zryzda22

Keywords:

Face Emotion Classification, Federated Learning, Deep Learning, Privacy Protection.

Abstract

Facial Emotion Recognition and Classification (FERC) is an important technology used in many areas, but it also brings up serious privacy worries because it depends on sensitive facial information. Federated Learning (FL) is a useful way to handle these issues because it lets different systems train a model together without actually sharing their private data. This paper provides a comprehensive review of the integration of FL with FERC. It surveys key FL methodologies--including horizontal, personalized, and cross-silo FL--applied to emotion classification using architectures such as CNNs and ResNet. The review also identifies critical challenges, such as data heterogeneity, the privacy-performance trade-off, and system heterogeneity, which impact model accuracy and practicality. Future research could focus more on personalized and semi-supervised federated learning, explore cross-modal learning, and look into using blockchain to make the systems more trustworthy. This study emphasizes how FL can help keep user data private while still working well in FERC systems. It’s a useful example for others looking to improve these kinds of systems in the future.

Downloads

Download data is not yet available.

References

[1] Capriola-Hall N N. Group Differences in Facial Emotion Expression in Autism: Evidence for the Utility of Machine Classification. Behavior Therapy, 2019, 50 (4): 828-838.

[2] Metzen E, Nayyeri D M, Schäfer R, et al. Emotion Processing of Facial Affect Expression in Patients with Somatic Symptom Disorder with Predominant Pain - An EEG-Study. NeuroImage, 2025, 307: 121036.

[3] Michele P D, Vincenzo O, Giovanna F, et al. Differences in Facial Emotion Recognition Between Bipolar Disorder and Other Clinical Populations: A Systematic Review and Meta-Analysis. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 2023, 127: 110847.

[4] Xiang G, Yao S, Wu X, et al. Driver Multi-Task Emotion Recognition Network Based on Multi-Modal Facial Video Analysis. Pattern Recognition, 2025, 161: 111241.

[5] Virolle J, Mouchet S, Robert L, et al. Facial Emotion Recognition in Sexual Offenders. Aggression and Violent Behavior, 2024, 78: 101982.

[6] Konečný J, McMahan H B, Yu F X, et al. Federated Learning: Strategies for Improving Communication Efficiency. arXiv, 2016.

[7] Abdulrahman S, Tout H, Ould-Slimane H, et al. A Survey on Federated Learning: The Journey from Centralized to Distributed On-Site Learning and Beyond. IEEE Internet of Things Journal, 2021, 8 (7): 5476–5497.

[8] Shome D, Kar T. FedAffect: Few-Shot Federated Learning for Facial Expression Recognition. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021: 4151–4158.

[9] Su X, Zhou Y, Cui L, et al. On Model Transmission Strategies in Federated Learning with Lossy Communications. IEEE Transactions on Parallel and Distributed Systems, 2023: 1–14.

[10] McMahan B, Moore E, Ramage D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017: 1273-1282.

[11] Arivazhagan M G, Aggarwal V, Singh A K, et al. Federated Learning with Personalization Layers. arXiv, 2019.

[12] Fallah A, Mokhtari A, Ozdaglar A. Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach. Advances in Neural Information Processing Systems, 2020, 33: 3557-3568.

[13] Li X, Jiang M, Zhang X, et al. Federated Meta-Learning for Facial Emotion Recognition. Proceedings of the Twelfth Language Resources and Evaluation Conference, 2020: 6522-6529.

[14] Huang Y, Du J, Yang Z, et al. Towards Federated Learning for Emotion Recognition from Mobile Device Sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2021, 5 (3): Article 156.

[15] Feng Y, Li B, Wang Y, et al. FedMSA: A Benchmark for Federated Multi-Modal Sentiment Analysis with Cross-Modal Self-Attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024: 1-10.

[16] Hsu T M H, Qi H, Brown M. Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification. arXiv, 2019.

[17] Pan K, Ong Y S, Gong M, et al. Differential Privacy in Deep Learning: A Literature Survey. Neurocomputing, 2024, 589: 127663.

[18] Wang H, Yurochkin M, Sun Y, et al. Federated Learning with Matched Averaging. arXiv, 2020.

[19] Smith V, Chiang C K, Sanjabi M, et al. Federated Multi-Task Learning. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), 2017: 4424-4434.

[20] Jeong W, Yoon J, Yang E, et al. FedMatch: Federated Learning Over Heterogeneous Data. Proceedings of the 37th International Conference on Machine Learning, 2020: 5136-5145.

[21] Han S, Cheng H, Zhang Y, et al. FedMultimodal: A Benchmark for Multimodal Federated Learning. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022: 9232-9250.

[22] Qin Y, Wang S, Yi S, et al. Blockchain-Based Federated Learning: A Comprehensive Survey. International Journal of Intelligent Systems, 2021, 36 (11): 8000-8025.