Dynamic Hyper-Parameter Adjustment in Fedprox for Improving Performance in Multi-Task Federated Learning Systems

Abstract

This paper presents FedSC-MTL, a unified framework for multi-task federated learning (MTL) that incorporates dynamic regularization and dual control stabilization to address key challenges such as data heterogeneity, task divergence, and knowledge misalignment. While existing methods like FedProx and SCAFFOLD focus on data heterogeneity and client drift, they fail to manage task-level goals and knowledge transfer across clients effectively. To overcome this, FedSC-MTL introduces a novel dual control mechanism: the control variable 'c' stabilizes parameter updates and mitigates client drift, while the feature-level distillation control 'd' enables efficient knowledge transfer. By focusing on feature-based distillation, FedSC-MTL reduces communication costs compared to logits-level distillation methods like FedICT and FedDyn, without sacrificing task-level generalization. The effectiveness of this approach is evaluated on multi-task federated benchmarks, including EMNIST, CIFAR-10, and Fashion-MNIST datasets. Experimental results show that FedSC-MTL outperforms existing methods in terms of accuracy, convergence speed, and communication efficiency. Specifically, the feature-level distillation mechanism significantly improves cross-task generalization, and the dual control stabilization enhances the overall stability and convergence of the model. These findings demonstrate the potential of FedSC-MTL as an effective solution for multi-task federated learning in heterogeneous environments, offering both scalability and efficiency.