Performance Analysis of Model-Contrastive Algorithm
DOI:
https://doi.org/10.54097/hset.v39i.6600Keywords:
Federated Learning; Image Classification; Deep Neural Network; Contrastive Learning; Machine Learning.Abstract
Federated learning develops multiple clients with a universal framework platform to train deep learning models with excellent performance. During this process, the local data set is not leaked to the outside, making federated learning a powerful method to protect data security while solving the problem of isolated data islands. However, in some usage scenarios, different clients are holding local data sets which are non- Independent Identically Distribution, i.e., non-IID. There have been significant attempts made to overcome this difficulty, and one of these efforts is the model-contrastive federated learning framework, as known as MOON algorithm. The main concept is to make use of the similarities between the global and local models to adjust the local training. The final global model performs well with non-IID data. The paper aims to replicate and analyze this algorithm to provide data reference for subsequent use. The algorithm is tested on the MNIST data set to get its performance on the handwritten digit classification task. This paper processed the data set to make it non-IID, but the result achieved a high accuracy rate with an acceptable computational cost.
Downloads
References
Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast-learning algorithm for deep belief nets. Neural computation, 18(7), 1527-1554.
Wang, J. Z., Kong, L. W., Huang, Z., et al. (2020). Research review of federated learning algorithms. Big Data Reserch, 6(6), 70-88.
Kairouz, P., McMahan, H. B., Avent, B., et al. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2), 1-210.
McMahan, B., Moore, E., Ramage, D., et al. (2017, April). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics (pp. 1273-1282).
Karimireddy, S. P., Kale, S., Mohri, M., et al. (2019). SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning.
Li, Q., He, B., & Song, D. (2021). Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10713-10722).
Li, T., Sahu, A. K., Zaheer, M., et al. (2020). Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2, 429-450.
He, K., Fan, H., Wu, Y., et al. (2020). Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9729-9738).
Chen, T., Kornblith, S., Norouzi, M., et al. (2020, November). A simple framework for contrastive learning of visual representations. In International conference on machine learning (pp. 1597-1607).
Deng, L. (2012). The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6), 141-142.
Downloads
Published
Issue
Section
License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.







