Proactive Service Composition Based on Deep Reinforcement Learning
DOI:
https://doi.org/10.54097/ymab3w50Keywords:
Service Composition, LSTM Neural Network, Deep Reinforcement LearningAbstract
With the development of cloud-network integration technology, how to compose cloud-network services with similar functions but inconsistent Quality of Service (QoS) to meet the diverse needs of users in different scenarios, such as space, air, land, and sea, has become an important research topic in the field of service computing. However, traditional service composition approaches ignore the importance of actively selecting the nearest server for service composition by perceiving user’s mobility, which may lead in delayed delivery of results to users after service composition is completed. Moreover, these approaches ignore the geographic spatial characteristics in the cloud-network environment. When facing a massive number of cloud-network services, individual servers may struggle to meet users' QoS requirements, leading to inefficient proactive service composition. To address these issues, this paper proposes an innovative service composition approach, called Proactive Service Composition Based on Deep Reinforcement Learning (PSCDRL). In this method, a Long Short-Term Memory (LSTM) models are constructed using user's location data to predict user trajectory, enabling the early acquisition of user positions and proactive select the nearest servers based on the predicted locations. In the process of service composition, according to the selected server, the deep reinforcement learning (DRL) method is used to perform service composition under the cooperation of multiple servers. The experiment demonstrated the feasibility and effectiveness of the method.
Downloads
References
[1] Billhardt, H., Hermoso, R., Ossowski, S., & Centeno, R. (2007). Trust based service provider selection in open environments. Proceedings of the 2007 ACM symposium on Applied computing, 1375 1380. https://doi.org/10. 1145/1244002. 1244298.
[2] Vakili, A., & Navimipour, N. J. (2017). Comprehensive and systematic review of the service composition mechanisms in the cloud environments. Journal of Network and Computer Applications, 81, 24 36. https://doi.org/ 10.1016/ j.jnca. 2017. 01. 005.
[3] Rao, J., & Su, X. (2004). A survey of automated web service composition methods. International workshop on semantic web services and web process composition, 43 54. Springer Berlin Heidelberg.
[4] Ismaeel, A. R., Moayad, A., Yehia, K., Yousif, A. R., & Yaser, J. (2018). A collaborative mobile edge computing and user solution for service composition in 5G systems. Transactions on Emerging Telecommunications Technologies, 29(11), e3446. https://doi.org/10.1002/ett.3446.
[5] Strunk, A. (2010). QoS aware service composition: A survey. 2010 Eighth IEEE European Conference on Web Services, 67 74. https://doi.org/10.1109/ECOWS.2010.18.
[6] Baresi, L., & Guinea, S. (2005). Towards dynamic monitoring of WS BPEL processes. Proceedings of the 3rd International Conference on Service Oriented Computing, 269 282. https:// doi.org/10.1007/11596141_21.
[7] Chan, K., Poernomo, I., Schmidt, H. W., & Jayaputera, J. (2005). A model oriented framework for runtime monitoring of nonfunctional properties. International Conference on the Quality of Software Architectures, 38 52. https://doi.org/ 10. 1007/ 11585597_4.
[8] Sammapun, U., Lee, I., Sokolsky, O., & Regehr, J. (2007). Statistical runtime checking of probabilistic properties. Proceedings of the 7th International Conference on Runtime Verification, 164 175. https://doi.org/ 10. 1007/978 3 5 40 73 520 0_11.
[9] Ardagna, D., & Pernici, B. (2007). Adaptive service composition in flexible processes. IEEE Transactions on Software Engineering, 33(6), 369 384. https://doi.org/ 10. 1109/ TSE.2007.1013.
[10] Beauche, S., & Poizat, P. (2008). Automated service composition with adaptive planning. Service Oriented Computing–ICSOC 2008: 6th International Conference, 530 537. Springer Berlin Heidelberg. https://doi.org/ 10. 1007/ 978 3 540 89648 3_41.
[11] da Silva, A. S., Ma, H., & Zhang, M. (2016). Genetic programming for QoS aware web service composition and selection. Soft Computing, 20(10), 3851 3867. https:// doi. org/ 10. 1007/s00500 015 1754 0.
[12] Wang, H., Zhou, X., Zhou, X., Li, W., & Bouguettaya, A. (2010). Adaptive service composition based on reinforcement learning. Service Oriented Computing: 8th International Conference, ICSOC 2010, 92 107. https://doi.org/ 10. 1007/ 97 8 3 642 17313 2_8.
[13] Ren, L., Wang, W., & Xu, H. (2017). A reinforcement learning method for constraint satisfied services composition. IEEE Transactions on Services Computing, 13(5), 786 800. https:// doi. org/ 10.1109/TSC.2017.2778740.
[14] Wang, H., Gu, M., Yu, Q., Fei, H., Li, J., & Tao, Y. (2017). Large scale and adaptive service composition using deep reinforcement learning. Service Oriented Computing: 15th International Conference, ICSOC 2017, 383 391. https://doi. org/ 10.1007/978 3 319 69035 3_27.
[15] Moustafa, A., & Ito, T. (2018). A deep reinforcement learning approach for large scale service composition. Proceedings of the International Conference on Principles and Practice of Multi Agent Systems, 296 311. Springer Verlag. https:// doi. org/ 10.1007/978 3 030 01771 2_17.
[16] Yu, Y., Si, X., Hu, C., & Zhang, J. (2019). A review of recurrent neural networks: LSTM cells and network architectures. Neural Computation, 31(7), 1235 1270. https:// doi. org/10.1162/neco_a_01199.
[17] Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2015). Human level control through deep reinforcement learning. Nature, 518(7540), 529 533. https://doi.org/10.1038/nature14236.
[18] Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing Atari with deep reinforcement learning [Preprint]. arXiv. https://arxiv. org/ abs/1312.5602.
[19] Xu, Z., Zou, G., Xia, X., Liu, Y., Gan, Y., Zhang, B., & He, Q. (2020). Distance aware edge user allocation with QoE optimization. 2020 IEEE International Conference on Web Services (ICWS), 66 74. https://doi.org/ 10.1109/ ICWS 48817. 2020.00019.
[20] Wu, H., Deng, S., Li, W., Yin, J., & Zomaya, A. Y. (2019). Mobility aware service selection in mobile edge computing systems. 2019 IEEE International Conference on Web Services (ICWS), 201 208. https://doi.org/10.1109/ICWS.2019.00038.
[21] Deng, S., Xiang, Z., Taheri, J., Khoshkholghi, M. A., Yin, J., Zomaya, A. Y., & Dustdar, S. (2021). Optimal application deployment in resource constrained distributed edges. IEEE Transactions on Mobile Computing, 20(5), 1907 1923. https://doi.org/10.1109/TMC.2020.2970698.
[22] Li, Y., Zhou, A., Ma, X., & Wang, S. (2022). Profit aware edge server placement. IEEE Internet of Things Journal, 9(1), 55 67. https://doi.org/10.1109/JIOT.2021.3082898.
[23] Guo, Y., Wang, S., Zhou, A., Xu, J., Yuan, J., & Hsu, C. H. (2020). User allocation aware edge cloud placement in mobile edge computing. Software: Practice and Experience, 50(5), 489 502. https://doi.org/10.1002/spe.2785.
[24] Wang, S., Guo, Y., Zhang, N., Yang, P., Zhou, A., & Shen, X. (2021). Delay aware microservice coordination in mobile edge computing: A reinforcement learning approach. IEEE Transactions on Mobile Computing, 20(3), 939 951. https:// doi. org/ 10.1109/TMC.2020.2991513.
[25] Luo, Z., Li, Y., & Yin, J. (2015). A Framework for Transmission Cost Aware Service Selection. 2015 IEEE International Conference on Web Services, 503 510. https://doi.org/10.1109/ICWS.2015.74.
[26] Luo, Z., Li, Y., & Yin, J. (2013). Location: A feature for service selection in the era of big data. 2013 IEEE 20th International Conference on Web Services, 515 522. https://doi.org/ 10. 1109/ ICWS.2013.77.
[27] Chen, F., Dou, R., Li, M., & Wu, H. (2016). A flexible QoS aware Web service composition method by multi objective optimization in cloud manufacturing. Computers & Industrial Engineering, 99, 423 431. https:// doi. org/ 10.1016/j.cie.2015.12.018.
[28] Wang, S., Zhao, Y., Huang, L., Xu, J., & Yuan, J. (2019). QoS prediction for service recommendations in mobile edge computing. Journal of Parallel and Distributed Computing, 127, 134 144. https://doi.org/10.1016/j.jpdc.2019.01.009.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
