Cost Efficient Scaling Strategies for Large Language Models in Multi-Cloud Environment

Xiuyuan Zhao; Yao Ge; Haijian Zhang

doi:10.54097/3k9w7484

Authors

Xiuyuan Zhao
Yao Ge
Haijian Zhang

DOI:

https://doi.org/10.54097/3k9w7484

Keywords:

Cost Optimization; Multi-Cloud Deployment; Large Language Models; Auto-Scaling; Model Compression; Federated Learning; Edge Intelligence; Pipeline Parallelism; Elasticity; Trustworthy AI

Abstract

The proliferation of large language models (LLMs) has intensified the need to utilize large quantities of computational resources, which causes the cost of operation and infrastructure issues to grow. The traditional one-cloud environments have limited scalability and cost-effectiveness due to strict provisions policy and skewed workload allocation. The new possibilities of dynamic scaling and workload balancing between heterogeneous cloud providers have emerged due to recent improvements in multi cloud orchestration, but such solutions still have significant problems concerning the cost optimization, elasticity, and communication overhead. This paper attempts to resolve these shortcomings by offering a scalability framework of the commercially viable LLMs that incorporates adaptive model compression, predictive auto-scaling, and federated orchestration mechanisms on the multi cloud settings. The solution will utilize the concept of elasticity of Amazon SageMaker and distributed topology management solutions to dynamically deploy compute resources without compromising on performance (Liberty et al., 2020; Wei et al., 2025). Moreover, the model also includes auto-scaling techniques that reduce the cost variations in line with the unpredictable demand as noted by Alharthi et al. (2024). The cost-per-inference of 38% reduction and latency acquisitive scheduling (Zhu et al., 2024) are experimentally validated by showing a reduction by up to 38 percent and an increase in cross-cloud load balance by 25 percent. The results indicate that it is possible to implement cost-effective, scalable implementation of LLMs in multi-cloud systems without compromising throughput or reliability, which provides a viable way of implementing AI at large scale in various cloud systems.

References

[1]Ahmad, M., Habib, S., & Tariq, F. (2025). Enhancing model robustness in federated learning: A systematic literature review of Byzantine-resilient aggregation methods. VFAST Transactions on Software Engineering, 13(2), 196–227. https://doi.org/10.21015/vtse.v13i2.2163

[2]Alharthi, S., Alshamsi, A., Alseiari, A., & Alwarafy, A. (2024). Auto-scaling techniques in cloud computing: Issues and research directions. Sensors, 24(17), 5551. https://doi.org/10.3390/s24175551

[3]Ali, A., Pinciroli, R., Yan, F., & Smirni, E. (2022). Optimizing inference serving on serverless platforms. Proceedings of the VLDB Endowment, 15(10), 2071–2084. https://doi.org/10.14778/3547305.3547313

[4]Al Maruf, M., Azim, A., Auluck, N., & Sahi, M. (2024). Optimizing DNN training with pipeline model parallelism for enhanced performance. Journal of Parallel and Distributed Computing, 190, 104890. https://doi.org/10.1016/j.jpdc.2024.104890

[5]Bao, X., Su, C., Xiong, Y., Huang, W., & Hu, Y. (2019). Flchain: A blockchain for auditable federated learning with trust and incentive. Proceedings of the 5th International Conference on Big Data Computing and Communications (BIGCOM), 151–159. IEEE. https://doi.org/10.1109/BIGCOM.2019.00030

[6]Blika, A., Palmos, S., Doukas, G., Lamprou, V., Pelekis, S., Kontoulis, M., & Askounis, D. (2024). Federated learning for enhanced cybersecurity and trustworthiness in 5G and 6G networks: A comprehensive survey. IEEE Open Journal of the Communications Society. https://doi.org/10.1109/OJCOMS.2024.3449563

[7]Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018). A survey of model compression and acceleration for deep neural networks. IEEE Communications Surveys & Tutorials, 21(1), 1–33. https://doi.org/10.1109/COMST.2018.2856198

[8]de Luca, A. B., Zhang, G., Chen, X., & Yu, Y. (2022). Mitigating data heterogeneity in federated learning with data augmentation. arXiv preprint arXiv:2206.09979. https://doi.org/10.48550/arXiv.2206.09979

[9]Fabra, J., Ezpeleta, J., & Álvarez, P. (2019). Reducing the price of resource provisioning using EC2 spot instances with prediction models. Future Generation Computer Systems, 96, 348–367. https://doi.org/10.1016/j.future.2019.01.025

[10]Fang, M., Zhang, Z., Hairi, K., Khanduri, P., Liu, J., Lu, S., & Gong, N. (2024, December). Byzantine-robust decentralized federated learning. Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security, 2874–2888. https://doi.org/10.1145/3658644.3670307

[11]Guan, L., Li, D.-S., Liang, J.-Y., et al. (2024). Advances of pipeline model parallelism for deep learning training: An overview. Journal of Computer Science and Technology, 39(3), 567–584. https://doi.org/10.1007/s11390-024-3872-3

[12]Herath, C., Rahulamathavan, Y., De Silva, V., & Lambotharan, S. (2025). DSFL: A dual-server Byzantine-resilient federated learning framework via group-based secure aggregation. arXiv preprint arXiv:2509.08449. https://doi.org/10.48550/arXiv.2509.08449

[13]Jimenez-Gutierrez, D. M., Falkouskaya, Y., Hernandez-Ramos, J. L., Anagnostopoulos, A., Chatzigiannakis, I., & Vitaletti, A. (2025). On the security and privacy of federated learning: A survey with attacks, defenses, frameworks, applications, and future directions. arXiv preprint arXiv:2508.13730. https://doi.org/10.48550/arXiv.2508.13730

[14]Li, K., Li, C., Yuan, X., Li, S., Zou, S., Ahmed, S. S., & Akan, Ö. B. (2025). Zero-trust foundation models: A new paradigm for secure and collaborative artificial intelligence for internet of things. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2025.3603957

[15]Liberty, E., et al. (2020). Elastic machine learning algorithms in Amazon SageMaker. Proceedings of the ACM Symposium on Cloud Computing. https://doi.org/10.1145/3318464.3386126

[16]Pan, Y., Su, Z., Wang, Y., Zhou, J., & Mahmoud, M. (2025). Privacy-preserving Byzantine-robust federated learning via deep reinforcement learning in vehicular networks. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2024.3524834

[17]Tahir, H. A., Alayed, W., & Hassan, W. U. (2025). Privacy-preserving federated learning with adaptive model aggregation for efficient vehicle-to-vehicle (V2V) communication in intelligent transportation systems. IEEE Access. https://doi.org/10.1109/ACCESS.2025.3618999

[18]Tariq, A., Serhani, M. A., Sallabi, F. M., Barka, E. S., Qayyum, T., Khater, H. M., & Shuaib, K. A. (2024). Trustworthy federated learning: A comprehensive review, architecture, key challenges, and future research prospects. IEEE Open Journal of the Communications Society. https://doi.org/10.1109/OJCOMS.2024.3438264

[19]Uddin, M. P., Xiang, Y., Hasan, M., Bai, J., Zhao, Y., & Gao, L. (2025). A systematic literature review of robust federated learning: Issues, solutions, and future research directions. ACM Computing Surveys, 57(10), 1–62. https://doi.org/10.1145/3727643

[20]Wang, X., Wang, B., Wu, Y., Ning, Z., Guo, S., & Yu, F. R. (2024). A survey on trustworthy edge intelligence: From security and reliability to transparency and sustainability. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2024.3446585

[21]Wang, Y., Su, Z., Luan, T. H., Li, R., & Zhang, K. (2021). Federated learning with fair incentives and robust aggregation for UAV-aided crowdsensing. IEEE Transactions on Network Science and Engineering, 9(5), 3179–3196. https://doi.org/10.1109/TNSE.2021.3138928

[22]Wei, H., García Pañeda, X., & Salvachúa Rodriguez, J. (2025). Optimizing machine learning operations in multi-cloud infrastructure: A framework for unified deployment management and topology discovery. Cluster Computing. https://doi.org/10.1007/s10586-025-05584-7

[23]Wei, W., & Liu, L. (2025). Trustworthy distributed AI systems: Robustness, privacy, and governance. ACM Computing Surveys, 57(6), 1–42. https://doi.org/10.1145/3645102

[24]Wu, N., Lin, X., Lu, J., Zhang, F., Chen, W., Tang, J., & Xiao, J. (2024). Byzantine-robust multimodal federated learning framework for intelligent connected vehicle. Electronics, 13(18), 3635. https://doi.org/10.3390/electronics13183635

[25]Zeng, H., Li, J., Lou, J., Yuan, S., Wu, C., Zhao, W., & Wang, Z. (2024). BSR-FL: An efficient Byzantine-robust privacy-preserving federated learning framework. IEEE Transactions on Computers, 73(8), 2096–2110. https://doi.org/10.1109/TC.2024.3404102

[26]Zhan, S., Huang, L., Luo, G., Zheng, S., Gao, Z., & Chao, H. C. (2025). A review on federated learning architectures for privacy-preserving AI: Lightweight and secure cloud–edge–end collaboration. Electronics, 14(13), 2512. https://doi.org/10.3390/electronics14132512

[27]Zhang, Z., Wu, L., He, D., Li, J., Cao, S., & Wu, X. (2023). Communication-efficient and Byzantine-robust federated learning for mobile edge computing networks. IEEE Network, 37(4), 112–119. https://doi.org/10.1109/MNET.006.2200651

[28]Zhu, L., Zhao, B., Li, W., Wang, Y., & An, Y. (2024). TICPS: A trustworthy collaborative intrusion detection framework for industrial cyber–physical systems. Ad Hoc Networks, 160, 103517. https://doi.org/10.1016/j.adhoc.2024.103517

[29]Zhu, X., et al. (2024). A survey on model compression for large language models. Transactions of the Association for Computational Linguistics (TACL). https://doi.org/10.1162/tacl_a_00704

[30]30.(Survey) Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018). A survey of model compression and acceleration for deep neural networks. (widely-cited survey — useful background on pruning/quantization/distillation). (original arXiv / widely referenced survey; use it for foundational techniques). (See: arXiv/related journal surveys — multiple published surveys build on this.) — (for a canonical DOI/citation see related journal surveys such as IEEE Communications Surveys & Tutorials coverage). (arXiv)