A Survey of Developments in Federated Meta-Learning

Yong Zhang; Mingchuan Zhang

doi:10.54097/bzpfwa11

Authors

Yong Zhang
Mingchuan Zhang

DOI:

https://doi.org/10.54097/bzpfwa11

Keywords:

Federated Learning; Meta-Learning; Federated Meta-Learning.

Abstract

Federated meta-learning is a widely used few-shot learning method and has a very good development prospect. Federated meta-learning combines the characteristics of federated learning and meta-learning. It can not only use the data of each client while protecting its privacy to a certain extent, but also solve the problem of data volume that requires a large amount of data for model training in machine learning. With the rise of big data technology and edge computing, federated meta-learning technology has become a research hotspot in machine learning. In this paper, we provide an overview of the development of federated meta-learning and point out the relationship between federated learning, meta-learning and federated learning. Finally, some existing problems in federated meta-learning are pointed out, which provides ideas for the subsequent research on federated meta-learning.

Downloads

Download data is not yet available.

References

Mjolsness E, DeCoste D. Machine learning for science: state of the art and future prospects [J]. Science, 2001, 293(5537): 2051-2055.

Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.

Moraveik M, Schmid M, Burch N, et al. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker[J]. Science, 2017, 356(6337): 508-513.

Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.

Sun P, Kretzschmar H, Dotiwalla X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 2446-2454.

Konečný J, McMahan H B, Ramage D, et al. Federated optimization: Distributed machine learning for on-device intelligence[J]. arXiv preprint arXiv:1610.02527, 2016.

Wang J, Tantia V, Ballas N, et al. Slowmo: Improving communication-efficient distributed sgd with slow momentum [J]. arXiv preprint arXiv:1910.00643, 2019.

Khanduri P, Sharma P, Yang H, et al. Stem: A stochastic two-sided momentum algorithm achieving near-optimal sample and communication complexities for federated learning[J]. Advances in Neural Information Processing Systems, 2021, 34: 6050-6061.

Gupta S, Ahuja K, Havaei M, et al. FL Games: A federated learning framework for distribution shifts[J]. arXiv preprint arXiv:2205.11101, 2022.

Yoon T, Shin S, Hwang S J, et al. Fedmix: Approximation of mixup under mean augmented federated learning[J]. arXiv preprint arXiv:2107.00233, 2021.

Thrun S, Pratt L. Learning to learn: Introduction and overview[M]. Learning to learn. Boston, MA: Springer US, 1998: 3-17.

Metz L, Maheswaranathan N, Cheung B, et al. Meta-learning update rules for unsupervised representation learning[J]. arXiv preprint arXiv:1804.00222, 2018.

Andrychowicz M, Denil M, Gomez S, et al. Learning to learn by gradient descent by gradient descent[J]. Advances in neural information processing systems, 2016, 29.

Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks[C]. International conference on machine learning. 2017: 1126-1135.

Donahue J, Jia Y, Vinyals O, et al. Decaf: A deep convolutional activation feature for generic visual recognition[C]. International conference on machine learning. 2014: 647-655.

Obamuyide A, Vlachos A. Model-agnostic meta-learning for relation classification with limited supervision[C]. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 5873-5879.

Chen F, Luo M, Dong Z, et al. Federated meta-learning with fast convergence and efficient communication[J]. arXiv preprint arXiv:1802.07876, 2018.

Khodak M, Balcan M F F, Talwalkar A S. Adaptive gradient-based meta-learning methods[J]. Advances in Neural Information Processing Systems, 2019, 32.

Fallah A, Mokhtari A, Ozdaglar A. Personalized federated learning: A meta-learning approach[J]. arXiv preprint arXiv:2002.07948, 2020.

Kayaalp M, Vlaski S, Sayed A H. Dif-MAML: Decentralized multi-agent meta-learning[J]. IEEE Open Journal of Signal Processing, 2022, 3: 71-93.

Mendieta M, Yang T, Wang P, et al. Local learning matters: Rethinking data heterogeneity in federated learning[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 8397-8406.

Caldarola D, Caputo B, Ciccone M. Improving generalization in federated learning by seeking flat minima[C]. European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 654-672.

Chaudhari P, Choromanska A, Soatto S, et al. Entropy-sgd: Biasing gradient descent into wide valleys[J]. Journal of Statistical Mechanics: Theory and Experiment, 2019, 2019(12): 124018.

Izmailov P, Podoprikhin D, Garipov T, et al. Averaging weights leads to wider optima and better generalization[J]. arXiv preprint arXiv:1803.05407, 2018.

Foret P, Kleiner A, Mobahi H, et al. Sharpness-aware minimization for efficiently improving generalization[J]. arXiv preprint arXiv:2010.01412, 2020.