A Comprehensive Analysis of Game theory on Multi-Agent Reinforcement
DOI:
https://doi.org/10.54097/gv6fpz53Keywords:
Multi-Agent, game theory, reinforcement, Nash EquilibriumAbstract
In recent years, reinforcement learning has gradually emerged as a popular research field in artificial intelligence. However, for multi-agent systems, due to their complex environment and an abundance of agents, the learning target of maximizing the anticipated value of cumulative rewards for specific agents frequently fails to converge. Introducing game theory into reinforcement learning can effectively address the interactions among intelligent agents, provide a rationale for convergence points corresponding to strategies. To address the issue of non-existence of optimal solutions for certain tasks in multi-agent settings, this paper takes a game-theoretic perspective and summarizes classical reinforcement learning algorithms developed in recent years. The basic theory of multi-agent reinforcement learning, essential game theory, categorization of reinforcement learning using multiple agents game strategies, primary worries are addressed in the study. It also analyzes the challenges that game-theoretic multi-agent reinforcement learning algorithms may encounter in the future, along with the relevant optimization directions.
Downloads
References
Sun Yu, CAO Lei, Chen Xiliang, et al. Review of multi-agent deep reinforcement learning [J]. Computer Engineering and Applications, 2020, 56(5):12.
Wang Jun, CAO Lei, Chen Xi liang. Review on reinforcement learning in multi-agent game [J]. Computer Engineering and Applications, 2021, 57(21):13.
Liu Hao, Research on reinforcement Learning Algorithm and its Equilibrium in Multi-Agent Game [D]. Xi 'an University of Science and Technology
Dorri A, Kanhere S S, Jurdak R. Multi-agent systems: A survey [J]. Ieee Access, 2018, 6: 28573-28593.
Buşoniu L, Babuška R, De Schutter B. Multi-agent reinforcement learning: An overview[J]. Innovations in multi-agent systems and applications-1, 2010: 183-221.
Solan E, Vieille N. Stochastic games [J]. Proceedings of the National Academy of Sciences, 2015, 112(45): 13743-13746.
Gronauer S, Diepold K. Multi-agent deep reinforcement learning: a survey[J]. Artificial Intelligence Review, 2022: 1-49.
Morgenstern O, Von Neumann J, Kuhn H W, et al. Theory of games and economic behavior [M]. J. Wiley and Sons, 1964.
Hu Yujing. Game, Equilibrium and Knowledge Transfer in Multi-agent reinforcement Learning [D]. Nanjing University.
Nash Jr J F. Equilibrium points in n-person games [J]. Proceedings of the national academy of sciences, 1950, 36(1): 48-49.
Wang X, Sandholm T. Reinforcement learning to play an optimal Nash equilibrium in team Markov games [J]. Advances in neural information processing systems, 2002, 15.
Arslan G, Yüksel S. Decentralized Q-learning for stochastic teams and games [J]. IEEE Transactions on Automatic Control, 2016, 62(4): 1545-1558.
Macua S V, Zazo J, Zazo S. Learning parametric closed-loop policies for markov potential games [J]. arXiv preprint arXiv:1802.00899, 2018.
Leslie D S, Collins E J. Generalised weakened fictitious play [J]. Games and Economic Behavior, 2006, 56(2): 285-298.
Mazumdar E, Ratliff L J, Sastry S. On the convergence of gradient-based learning in continuous games [J]. arXiv preprint arXiv:1804.05464, 2018.
Chen G. A New Framework for Multi-Agent Reinforcement Learning--Centralized Training and Exploration with Decentralized Execution via Policy Distillation [J]. arXiv preprint arXiv:1910.09152, 2019.
Adler I. The equivalence of linear programs and zero-sum games[J]. International Journal of Game Theory, 2013, 42: 165-177.
Chen X, Deng X. Settling the Complexity of Two-Player Nash Equilibrium [C]// IEEE Annual Symposium on Foundations of Computer Science. 2006, 6: 261-272.
Zinkevich M, Johanson M, Bowling M, et al. Regret minimization in games with incomplete information [J]. Advances in neural information processing systems, 2007, 20.
Bai Y, Jin C. Provable self-play algorithms for competitive reinforcement learning[C]//International conference on machine learning, 2020: 551-560.
Adler I. The equivalence of linear programs and zero-sum games [J]. International Journal of Game Theory, 2013, 42: 165-177.
Goodfellow I. Pouget⁃Abadie J, Mirza M, et al [J]. Generative adversarial nets, 2014, 2672: 2680.
Li S, Wu Y, Cui X, et al. Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient[C]//Proceedings of the AAAI conference on artificial intelligence. 2019, 33(01): 4213-4220.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Highlights in Science, Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
