From Verifiable Rewards to Autonomous Evolution: Reinforcement Learning-Driven Large Language Model Reasoning Abilities

Mengbo  Song

doi:10.54097/mzt6hz16

Authors

Mengbo Song Software Engineering, Northwestern Polytechnical University, Xi'an, Shaanxi, China

DOI:

https://doi.org/10.54097/mzt6hz16

Keywords:

Large Language Models, Reinforcement Learning, Verifiable Rewards, Logical Reasoning.

Abstract

This paper attempts to outline the evolution of LLMs from classic methods of supervised finetuning models with static human-annotated datasets, to a more dynamic and evolutionary reinforcement learning based autonomous models. Along with the systematic evolution of reasoning capabilities, this is one of the most prominent focal points in the field of AI. This paper attempt to outline the most current advancements in reasoning and reinforcement learning, and classify the occurrences into the applicable areas of criteria, such as: choice of architecture, choice of reward assignments, and choice of evaluation metrics. This paper explore reasoning improvements as a result of self-reflective and exploratory processes within the bounds of the RL with Verifiable Rewards (RLVR) framework. By systematically studying the aforementioned criteria across multiple works and the respective variations in algorithmic efficiency, control of reward signal bias, and performance metrics, this paper want to outline the positive role of reinforcement learning in fostering autonomous self-correction in models and complex thought chain processes. The aim of this paper is to describe the potential autonomous advancements the next generations of large language models may evolve and want to offer some suggestions as a theoretical and a technical framework.

References

[1] Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., ... & Tan, Y. (2025). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081), 633-638.

[2] Gao, J., Xu, S., Ye, W., Liu, W., He, C., Fu, W., ... & Wu, Y. (2024). On designing effective rl reward at training time for llm reasoning. arXiv preprint arXiv:2410.15115.

[3] Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Song, S., & Huang, G. (2025). Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. arXiv preprint arXiv:2504.13837.

[4] Cheng, Z., Hao, S., Liu, T., Zhou, F., Xie, Y., Yao, F., ... & Hu, Z. (2025). Revisiting reinforcement learning for llm reasoning from a cross-domain perspective. arXiv preprint arXiv:2506.14965.

[5] Xie, T., Gao, Z., Ren, Q., Luo, H., Hong, Y., Dai, B., ... & Luo, C. (2025). Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768.

[6] Chen, M., Sun, L., Li, T., Sun, H., Zhou, Y., Zhu, C., ... & Chen, W. (2025). Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470.

[7] Wen, X., Liu, Z., Zheng, S., Ye, S., Wu, Z., Wang, Y., ... & Yang, M. (2025). Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. arXiv preprint arXiv:2506.14245.

[8] Su, Y., Yu, D., Song, L., Li, J., Mi, H., Tu, Z., ... & Yu, D. (2025). Crossing the reward bridge: Expanding rl with verifiable rewards across diverse domains. arXiv preprint arXiv:2503.23829.

[9] Pan, P. C., Liang, Y., & Lin, S. (2026). Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation. arXiv preprint arXiv:2602.09305.

[10] Wei, Y., Duchenne, O., Copet, J., Carbonneaux, Q., Zhang, L., Fried, D., ... & Wang, S. I. (2025). Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502.18449.