Dynamic Hyper-Parameter Adjustment in Fedprox for Performance Optimization in Federated Learning

Abstract

This paper investigates the application of dynamic hyper-parameter adjustment strategies to improve the performance of the FedProx algorithm in federated learning (FL). FedProx addresses the challenges posed by data heterogeneity across clients, a common issue in FL that hinders convergence. While FedProx improves upon the basic FedAvg algorithm by introducing a proximal term to stabilize model updates, the hyper-parameter μ that controls the impact of the proximal term is sensitive and needs careful tuning. An inappropriate μ can degrade performance, making dynamic adjustment crucial. The study introduces two dynamic μ adjustment strategies. The first strategy, called ExponentialDecay, progressively reduces μ exponentially by multiplying it with a constant factor. The second strategy, AdaptiveLoss, adjusts μ based on changes in the loss value, decreasing it when the loss decreases and increasing it when the loss increases, offering a more responsive approach to the model’s performance. The effectiveness of both strategies is evaluated on the EMNIST dataset, which simulates real-world non-IID data distributions. Results show that both dynamic strategies outperform the static μ setting in terms of model accuracy, loss reduction, and training speed, particularly in scenarios with highly heterogeneous data. Specifically, the AdaptiveLoss strategy achieves the best performance when the data is extremely non-IID, reducing loss by 10.2% and improving accuracy by 2.5% compared to the static μ approach. These dynamic strategies not only enhance FedProx’s stability but also accelerate the convergence of the global model.