Research on Path Planning of Warehouse Robots Based on Q-learning

Abstract

The rapid growth of e-commerce has intensified the demand for efficient warehouse management and automation. This research focuses on applying Q-learning, a reinforcement learning algorithm, to the path planning of warehouse robots. Path planning is a critical aspect of warehouse robotics, particularly in dynamic environments where efficient navigation can significantly enhance operational productivity. This study aims to develop a Q-learning-based algorithm that enables warehouse robots to navigate their workspace effectively, minimizing the time taken to reach designated goals while avoiding obstacles. The approach involves constructing a grid world model to simulate the warehouse environment, where the robot learns to make optimal decisions based on rewards and penalties associated with reaching goals and colliding with obstacles, respectively. By modeling the warehouse as a discrete grid with obstacles and defining the robot's task cycle—from starting point to cargo pickup to delivery, and return—this study investigates how varying the learning rate (α) and discount factor (γ) affects the robot's learning efficiency and path optimization. A full-factor experiment testing 81 combinations of α and γ revealed that higher values of γ, especially when paired with appropriate α values, significantly reduce the number of steps required for task completion and decrease path-planning failures. The findings underscore the importance of parameter tuning in Q-learning algorithms to enhance the efficiency and reliability of autonomous warehouse robots.