Path Planning in Dynamic Environments Based on Q-Learning
DOI:
https://doi.org/10.54097/hset.v63i.10880Keywords:
Q-learning, Path planning, Dynamic obstacle avoidance, Reinforcement learning.Abstract
With the rapid progress of science and technology, the scope of applications for mobile robots is growing. Path planning in dynamic environment is always a challenging task for mobile robot, which shows significant impacts in the medical field and the military field. Q-learning, a model-free reinforcement learning algorithm, can recognize its surroundings and demonstrate a system making decisions for itself about how this algorithm learns to make accurate decisions about achieving optimal target. Therefore, a path planning algorithm was proposed for robots in dynamic environments based on Q-learning. This algorithm can successfully generate several viable paths in environments with both static obstacles, dynamic obstacles and target point. First, three environments with different levels of complexity were created. Afterwards, to generate several optimal paths, the path planning algorithm was conducted in multiple times in different environments. Finally, the experimental results were collected and visualized to illustrate information. The effectiveness of the proposed algorithm is validated by experiment results in solving problems of path planning in dynamic environments with required point.
Downloads
References
R. D. Field, P. N. Anandakumaran, and S. K. Sia, “Soft medical microrobots: Design components and system integration,” Applied physics reviews, vol. 6, p. 041305, 2019.
L. Piardi, J. Lima, A. Pereira, and P. Costa, “Coverage path planning optimization based on q-learning algorithm,” AIP Conference Proceedings, vol. 2116, 07 2019.
Delling, Daniel, et al. “Engineering route planning algorithms.” Algorithmics of large and complex networks. Springer, Berlin, Heidelberg, 2009. 117-139.
E. S. Low, P. Ong, and K. C. Cheah, “Solving the optimal path planning of a mobile robot using improved q-learning,” Robotics Auton. Syst., vol. 115, pp. 143–161, 2019.
C. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, pp. 279–292, 1992.
R. S. Sutton and A. G. Barto, “Reinforcement learning: An introduction,” IEEE Transactions on Neural Networks, vol. 16, pp. 285–286, 2005.
E. S. Low, P. Ong, C. Y. Low, and R. B. Omar, “Modified q-learning with distance metric and virtual target on path planning of mobile robot,” Expert Syst. Appl., vol. 199, p. 117191, 2022.
J. M. López-Guede, B. Fernández-Gauna, and M. Graña, “State-action value function modeled by elm in reinforcement learning for hose control problems,” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 21, pp. 99–116, 2013.
Y. Hu, L. Yang, and Y. Lou, “Path Planning with Q-Learning,” Journal of Physics: Conference Series, vol. 1948, p. 012038, June 2021. Publisher: IOP Publishing.
A. Maoudj and A. Hentout, “Optimal path planning approach based on Q-learning algorithm for mobile robots,” Applied Soft Computing, vol. 97, p. 106796, 2020.
B. Hao, H. Du, and Z. Yan, “A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning,” Ocean Engineering, vol. 270, p. 113632, 2023.
Y. Li, “Deep reinforcement learning: An overview,” ArXiv, vol. abs/1701.07274, 2017.
P. Chen, J. Pei, W. Lu, and M. Li, “A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance,” Neurocomputing, vol. 497, pp. 64–75, 2022.
M. L. Littman, “Markov games as a framework for multi-agent reinforcement learning,” in International Conference on Machine Learning, 1994.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
