Based on Hierarchical Reinforcement Learning for Large-scale Pedestrian Trajectory Prediction

Guokang Gao; Yue Li

doi:10.54097/PTsXludb

Authors

Guokang Gao
Yue Li

DOI:

https://doi.org/10.54097/PTsXludb

Keywords:

Hierarchical Reinforcement Learning, Reinforcement Learning, Pedestrian Trajectory, Anylogic Simulation Software

Abstract

This paper describes the construction of an airport terminal simulation model using AnyLogic simulation software. It considers the advantages of hierarchical reinforcement learning and divides the complete process of pedestrian trajectories at the airport into layers. Pedestrians are treated as intelligent agents for hierarchical reinforcement learning. A large-scale pedestrian trajectory planning algorithm based on hierarchical reinforcement learning is designed to match the hotspots in the airport region simulated by pedestrian trajectories with congested areas in the terminal scene. A comparison is made with traditional multi-agent Q-learning algorithms and single-table hierarchical reinforcement learning. The results show that our algorithm can accurately identify the pedestrian flow hotspots in the actual terminal, with improved matching accuracy compared to traditional multi-agent Q-learning algorithms and single-table hierarchical reinforcement learning. The algorithm also exhibits faster convergence speed.

Downloads

Download data is not yet available.

References

Canese, Lorenzo, Cardarilli, Gian, Carlo, Di, Nunzio, Luca, Fazzolari, Rocco, Giardino, Daniele, Re, Marco, Spano ,& Sergio. Multi-Agent Reinforcement Learning: A Review of Challenges and Applications[J].APPLIED SCIENCES-BASEL, 2021, 11 (11).

Peysakhovich, Alexander, Lerer, & Adam. Prosocial learning agents solve generalized stag hunts better than selfish ones [J]. Ar v,2017.

Zhang, Yaping,Li, Jialin,Kong, Dexuan, ng, aoqing,Luo, Qian, Mao, & Jian.Modeling and Simulation of Departure Passenger's Behavior Based on an Improved Social Force Approach: A Case Study on an Airport Terminal in China [J]. ADVANCES IN CIVIL ENGINEERING,2021,2021.

Mekic, Adin, Mohammadi, Ziabari, Seyed, Sahand, Sharpanskykh, & Alexei. Systemic Agent-Based Modeling and Analysis of Passenger Discretionary Activities in Airport Terminals [J]. AEROSPACE, 2021,8(6).

Rozema, LM. “Behavioural Classification of Passengers in an Airport Terminal.” (2017).

Al-Sultan, A. T. (2018). Simulation and Optimization for Modeling the Passengers Check-in System at Airport Terminal. Review of Integrative Business and Economics Research, 7(1), 44.

Shakur, Md. Ashab & Hasan, Muhammad. (2019). SIMULATION MODELING AND OPTIMIZATION OF THE CHECK-IN PROCESS OF AN INTERNATIONAL AIRPORT.

Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 6382–6393.

Iqbal S, Sha F. Actor-Attention-Critic for Multi-Agent Reinforcement Learning[J]. 2018.

Sutton R S, Precup D, Singh S. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning - ScienceDirect[J]. Artificial Intelligence, 1999, 112 (1–2):181-211.

Parr R E. Hierarchical Control and Learning for Markov Decision Processes[D]. University of California, 1998: 36-47.

Dietterich T G. An Overview of MAXQ Hierarchical Reinforcement Learning[M]. Abstraction, Reformulation, and Appro mation. Springer Berlin Heidelberg, 2000: 26-44.

Silver D , Lever G , Heess N , et al. Deterministic Policy Gradient Algorithms[C]// International Conference on Machine Learning. PMLR, 2014.

Lillicrap T P , Hunt J J , Pritzel A , et al. Continuous control with deep reinforcement learning[J]. Computer ence, 2015.

Curcio, Duilio et al. “Passengers’ Flow Analysis And Security Issues In Airport Terminals Using Modeling & Simulation.” (2007).