A Vehicle Service Migration Strategy Algorithm in 5G NR-V2X

Lv Zhou

doi:10.54097/fcis.v2i3.5211

Authors

Lv Zhou

DOI:

https://doi.org/10.54097/fcis.v2i3.5211

Keywords:

Mobile edge computing, Vehicle edge computing, Service migration, Markov Decision Process, Delay, Energy consumption, Deep reinforcement learning

Abstract

In recent years, a new technology called mobile edge computing (MEC) is proposed to alleviate mobile devices’ increasing demands on computing resources and service latency. However, users may stay away from the fixed MEC server or even leave the service scope of the MEC server during the mobile process, which will lead to service performance degradation or service interruption. Such an interruption is intolerable for vehicle in V2X. Therefore, it is necessary to consider the issue of service migration. In this paper, we model the service migration problem in V2X as a multidimensional Markov decision process problem and consider a variety of metrics including computing cost, migration cost and energy consumption. A novel user mobility model is also considered to accurately reflect the motion state of users in reality. Then, a novel service migration algorithm is proposed to solve this multidimensional MDP problem. Our proposed algorithm applies Actor-Critic and entropy to ensure both convergence and exportability. Experimental results show that our proposed algorithm outperforms the baselines and has relatively strong robustness.

Downloads

Download data is not yet available.

References

A. Kostopoulos, I. Chochliouros, J. Ferragut, Y. Ma, M. Kutila, A. Gavras, S. Horsmanheimo, K. Zhang, L. Ladid, and A. Dardamanis, "Use cases and standardisation activities for eMBB and V2X scenarios." pp. 1-7.

M. H. C. Garcia, A. Molina-Galan, M. Boban, J. Gozalvez, B. Coll-Perales, T. Şahin, and A. Kousaridas, "A tutorial on 5G NR V2X communications," IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1972-2026, 2021.

J. Bi, H. Yuan, S. Duanmu, M. Zhou, and A. Abusorrah, "Energy-optimized partial computation offloading in mobile-edge computing with genetic simulated-annealing-based particle swarm optimization," IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3774-3785, 2020.

X. Chen, H. Zhang, C. Wu, S. Mao, Y. Ji, and M. Bennis, "Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning," IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4005-4018, 2018.

T. Ouyang, Z. Zhou, and X. Chen, "Follow me at the edge: Mobility-aware dynamic service placement for mobile edge computing, " IEEE Journal on Selected Areas in Communications, vol. 36, no. 10, pp. 2333-2345, 2018.

H. D. R. Albonda, and J. Pérez-Romero, "An efficient RAN slicing strategy for a heterogeneous network with eMBB and V2X services, " IEEE access, vol. 7, pp. 44771-44782, 2019.

K. Guan, G. Li, T. Kürner, A. F. Molisch, B. Peng, R. He, B. Hui, J. Kim, and Z. Zhong, "On millimeter wave and THz mobile radio channel for smart rail mobility," IEEE Transactions on Vehicular Technology, vol. 66, no. 7, pp. 5658-5674, 2016.

G. Naik, B. Choudhury, and J.-M. Park, "IEEE 802.11 bd & 5G NR V2X: Evolution of radio access technologies for V2X communications, " IEEE access, vol. 7, pp. 70169-70184, 2019.

M. Harounabadi, D. M. Soleymani, S. Bhadauria, M. Leyh, and E. Roth-Mandutz, "V2X in 3GPP standardization: NR sidelink in release-16 and beyond," IEEE Communications Standards Magazine, vol. 5, no. 1, pp. 12-21, 2021.

B. Sklar, "Rayleigh fading channels in mobile digital communication systems .I. Characterization," IEEE Communications Magazine, vol. 35, no. 7, pp. 90-100, 1997.

J. Wildman, P. H. J. Nardelli, M. Latva-aho, and S. Weber, "On the Joint Impact of Beamwidth and Orientation Error on Throughput in Directional Wireless Poisson Networks," IEEE Transactions on Wireless Communications, vol. 13, no. 12, pp. 7072-7085, 2014.

Z. Li, L. Xiang, X. Ge, G. Mao, and H. C. Chao, "Latency and Reliability of mmWave Multi-Hop V2V Communications Under Relay Selections, " IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 9807-9821, 2020.

Y. Oike, M. Ikeda, and K. Asada, "A high-speed and low-voltage associative co-processor with exact Hamming/Manhattan-distance estimation using word-parallel and hierarchical search architecture," IEEE Journal of Solid-State Circuits, vol. 39, no. 8, pp. 1383- 1387, 2004.

L. Qing, "A 5G PaaS Collaborative Management and Control Platform Technology Based on Cloud Edge Collaboration Based on Particle Swarm Optimization Algorithm." pp. 144-147.

A. Sergeev, E. Rezedinova, and A. Khakhina, "Docker Container Performance Comparison on Windows and Linux Operating Systems." pp. 1-4.

R. Bellman, "A Markovian Decision Process," Journal of Mathematics and Mechanics, vol. 6, no. 5, pp. 679-684, 1957.

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor," in Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, 2018, pp. 1861--1870.

C. Banerjee, Z. Chen, and N. Noman, "Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples With On-Policy Experiences, " IEEE Transactions on Neural Networks and Learning Systems, 2022.

R. S. Sutton, and A. G. Barto, Reinforcement learning: an introduction: MIT press, 2018.

H. Van Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double q-learning."

S. Kullback, and R. A. Leibler, "On information and sufficiency," The annals of mathematical statistics, vol. 22, no. 1, pp. 79-86, 1951.