Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach
DOI:
https://doi.org/10.54097/646rv696Keywords:
Asynchronous Federated Learning · Machine Learning · Distributed Computing · Privacy and Security.Abstract
Federated Learning (FL) is a distributed machine learning paradigm that allows clients to train models on their data while preserving their privacy. FL algorithms, such as Federated Averaging (FedAvg) and its variants, have been shown to converge well in many scenarios. However, these methods require clients to upload their local updates to the server in a synchronous manner, which can be slow and unreliable in realistic FL settings. To address this issue, researchers have developed asynchronous FL methods that allow clients to continue training on their local data using a stale global model. However, most of these methods simply aggregate all of the received updates without considering their relative contributions, which can slow down convergence. In this paper, we propose a contribution-aware asynchronous FL method that takes into account the staleness and statistical heterogeneity of the received updates. Our method dynamically adjusts the contribution of each update based on these factors, which can speed up convergence compared to existing methods.
References
Ye, J., Kang, H., Wang, H., Shen, C., Jahannia, B., Heidari, E., ... & Dalir, H. (2023, September). Demultiplexing OAM beams via Fourier optical convolutional neural network. In Laser Beam Shaping XXIII (Vol. 12667, pp. 16-33). SPIE.
Ye, J., Solyanik, M., Hu, Z., Dalir, H., Nouri, B. M., & Sorger, V. J. (2023, March). Free-space optical multiplexed orbital angular momentum beam identification system using Fourier optical convolutional layer based on 4f system. In Complex Light and Optical Forces XVII (Vol. 12436, pp. 70-80). SPIE.
Ye, J., Kang, H., Wang, H., Altaleb, S., Heidari, E., Asadizanjani, N., ... & Dalir, H. (2023, October). OAM beams multiplexing and classification under atmospheric turbulence via Fourier convolutional neural network. In Frontiers in Optics (pp. JTu4A-73). Optica Publishing Group.
Ye, J., Kang, H., Wang, H., Altaleb, S., Heidari, E., Asadizanjani, N., ... & Dalir, H. (2023, November). Multiplexed OAM beams classification via Fourier optical convolutional neural network. In 2023 IEEE Photonics Conference (IPC) (pp. 1-2). IEEE.
Kang, H., Ye, J., Wang, H., Dalir, H., & Sorger, V. J. (2023, November). Optical Freespace Michelson Interferometric Reconfigurable Full Complex Convolution Module. In 2023 IEEE Photonics Conference (IPC) (pp. 1-2). IEEE.
Kang, H., Solyanik-Gorgone, M., Ye, J., Nouri, B. M., Wang, H., Dalir, H., & Sorger, V. J. (2023, November). Secure Optical Hashing for Information Compression in a Convolutional Neural Network. In 2023 IEEE Photonics Conference (IPC) (pp. 1-2). IEEE..
Wang, L., Bian, J., & Xu, J. (2023). Federated Learning with Instance-Dependent Noisy Labels. arXiv preprint arXiv: 2312.10324.
Bian, J., Ren, S., & Xu, J. (2023). CAFE: Carbon-Aware Federated Learning in Geographically Distributed Data Centers. arXiv preprint arXiv:2311.03615.
Bian, J., & Xu, J. (2023, October). Client Clustering for Energy-Efficient Clustered Federated Learning in Wireless Networks. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing (pp. 718-723).
Bian, J., Shen, C., & Xu, J. (2023). Joint Client Assignment and UAV Route Planning for Indirect-Communication Federated Learning. arXiv preprint arXiv:2304.10744.
Liu, T., Xu, C., Qiao, Y., Jiang, C., & Yu, J. (2024). Particle Filter SLAM for Vehicle Localization. arXiv preprint arXiv:2402.07429.
Liu, T., Xu, C., Qiao, Y., Jiang, C., & Chen, W. (2024). News Recommendation with Attention Mechanism. arXiv preprint arXiv:2402.07422.
Guo, C., Zhao, Y., Liu, T., & Yang, C. (2023). THE ROLE OF MACHINE LEARNING IN ENHANCING COMPUTER VISION PROCESSING. In АКТУАЛЬНЫЕ ВОПРОСЫ СОВРЕМЕННЫХ НАУЧНЫХ ИССЛЕДОВАНИЙ (pp. 23-26).
Popokh, L., Su, J., Nair, S., & Olinick, E. (2021, September). IllumiCore: Optimization Modeling and Implementation for Efficient VNF Placement. In 2021 International Conference on Software, Telecommunications and Computer Networks (SoftCOM) (pp. 1-7). IEEE.
Su, J., Nair, S., & Popokh, L. (2023, February). EdgeGym: A Reinforcement Learning Environment for Constraint-Aware NFV Resource Allocation. In 2023 IEEE 2nd International Conference on AI in Cybersecurity (ICAIC) (pp. 1-7). IEEE.
Su, J., Nair, S., & Popokh, L. (2022, November). Optimal Resource Allocation in SDN/NFV-Enabled Networks via Deep Reinforcement Learning. In 2022 IEEE Ninth International Conference on Communications and Networking (ComNet) (pp. 1-7). IEEE.
Xu, J., & Sen, S. (2023). Ensemble Variance Reduction Methods for Stochastic Mixed-Integer Programming and their Application to the Stochastic Facility Location Problem. INFORMS Journal on Computing.
Xu, J., & Sen, S. (2023). Compromise policy for multi-stage stochastic linear programming: Variance and bias reduction. Computers & Operations Research, 153, 106132.
Xu, J., & Sen, S. (2021). Decision Intelligence for Nationwide Ventilator Allocation During the COVID-19 Pandemic. SN Computer Science, 2(6), 423.
Deng, Y., Kesselman, C., Sen, S., & Xu, J. (2019, December). Computational operations research exchange (core): A cyber-infrastructure for analytics. In 2019 Winter Simulation Conference (WSC) (pp. 3447-3456). IEEE.
Liu, S., Wu, K., Jiang, C., Huang, B., & Ma, D. (2023). Financial Time-Series Forecasting: Towards Synergizing Performance And Interpretability Within a Hybrid Machine Learning Approach. arXiv preprint arXiv:2401.00534.
Xiong, J., Feng, M., Wang, X., Jiang, C., Zhang, N., & Zhao, Z. (2024). Decoding Sentiments: Enhancing COVID-19 Tweet Analysis through BERT-RCNN Fusion. Journal of Theory and Practice of Engineering Science, 4(01), 86-93.
Xu, C., Yu, J., Chen, W., & Xiong, J. (2024, January). DEEP LEARNING IN PHOTOVOLTAIC POWER GENERATION FORECASTING: CNN-LSTM HYBRID NEURAL NETWORK EXPLORATION AND RESEARCH. In The 3rd International scientific and practical conference “Technologies in education in schools and universities”(January 23-26, 2024) Athens, Greece. International Science Group. 2024. 363 p. (p. 295).
Zang, H. (2024). Precision Calibration of Industrial 3D Scanners: An AI-Enhanced Approach for Improved Measurement Accuracy. Global Academic Frontiers, 2(1), 27-37.
Zang, H., & Dong, X. (2024). Optimizing Soil Health Management in Smart Agriculture: Deep Learning Algorithms for Nutrient Analysis and Fertilizer Recommendation with Precision Agriculture Systems. Journal of Industrial Engineering and Applied Science, 2(1), 1-7.
Qiao, Y., Jin, J., Ni, F., Yu, J., & Chen, W. (2023). APPLICATION OF MACHINE LEARNING IN FINANCIAL RISK EARLY WARNING AND REGIONAL PREVENTION AND CONTROL: A SYSTEMATIC ANALYSIS BASED ON SHAP. WORLD TRENDS, REALITIES AND ACCOMPANYING PROBLEMS OF DEVELOPMENT, 331.
Qiao, Y., Ni, F., Xia, T., Chen, W., & Xiong, J. (2024, January). AUTOMATIC RECOGNITION OF STATIC PHENOMENA IN RETOUCHED IMAGES: A NOVEL APPROACH. In The 1st International scientific and practical conference “Advanced technologies for the implementation of new ideas”(January 09-12, 2024) Brussels, Belgium. International Science Group. 2024. 349 p. (p. 287).
Ni, F., Zang, H., & Qiao, Y. (2024, January). SMARTFIX: LEVERAGING MACHINE LEARNING FOR PROACTIVE EQUIPMENT MAINTENANCE IN INDUSTRY 4.0. In The 2nd International scientific and practical conference “Innovations in education: prospects and challenges of today”(January 16-19, 2024) Sofia, Bulgaria. International Science Group. 2024. 389 p. (p. 313).
Xiaobin, L., Changlin, Y., & Wanting, W. (2023). CARASSISTNET: DESIGN AND IMPLEMENTATION OF A DRIVING ASSISTANCE SYSTEM USING COMPUTER VISION. Ответственный редактор, 28.
Hu, W., Liu, X., & Xie, Z. (2022). Ore image segmentation application based on deep learning and game theory. In World science: problems and innovations (pp. 71-76).
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
