Navigating Complexity: GPT-4's Performance in Predicting Earnings and Stock Returns in China's A-Share Market
DOI:
https://doi.org/10.54097/4rwdat95Keywords:
Artificial Intelligence, GPT-4, Financial Analysis, China A-Share Market, Earnings Prediction, Stock Returns, Large Language Models, Market Efficiency, AI Ethics in Finance.Abstract
This study investigates the application of GPT-4, a large language model, in predicting earnings changes and stock returns within China's A-share market from 2000 to 2023. We evaluate the model's performance using various metrics, including prediction accuracy, F1 score, stock returns, Sharpe ratio, and alpha. Our findings reveal significant fluctuations in the model's predictive accuracy, ranging from 10.62% to 48.67%, with an average F1 score of 0.30. Despite inconsistent accuracy, the model maintained high prediction confidence levels between 75% and 90%. Stock returns associated with the model's predictions varied widely, from -4.86% to 13.59%, showing no consistent correlation with prediction accuracy. The study highlights the challenges of applying AI models to financial analysis in emerging markets, particularly given the unique characteristics of China's A-share market, such as frequent policy interventions and a high proportion of retail investors. We discuss the implications of these findings for the future of AI-driven financial analysis, emphasizing the need for improved model calibration, ethical considerations, and regulatory frameworks.
Downloads
References
[1] Bhat, R., & Jain, B. (2024, June). Stock Price Trend Prediction using Emotion Analysis of Financial Headlines with Distilled LLM Model. In Proceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments (pp. 67-73). DOI: https://doi.org/10.1145/3652037.3652076
[2] Cao, Y., Chen, Z., Pei, Q., Dimino, F., Ausiello, L., Kumar, P., ... & Ndiaye, P. M. (2024). RiskLabs: Predicting Financial Risk Using Large Language Model Based on Multi-Sources Data. arXiv preprint arXiv:2404.07452.
[3] Cao, Y., Chen, Z., Pei, Q., Kumar, P., Subbalakshmi, K. P., & Ndiaye, P. M. (2024). ECC Analyzer: Extract Trading Signal from Earnings Conference Calls using Large Language Model for Stock Performance Prediction. arXiv preprint arXiv:2404.18470. DOI: https://doi.org/10.1145/3677052.3698689
[4] Chen, S., Green, T. C., Gulen, H., & Zhou, D. (2024). What Does ChatGPT Make of Historical Stock Returns? Extrapolation and Miscalibration in LLM Stock Return Forecasts. arXiv preprint arXiv:2409.11540. DOI: https://doi.org/10.2139/ssrn.4941906
[5] Chen, Y., Kelly, B. T., & Xiu, D. (2022). Expected returns and large language models. Available at SSRN 4416687.
[6] Chiu, I., & Hung, M. W. Finance-Specific Large Language Models: Advancing Sentiment Analysis and Return Prediction with Llama 2. Mao-Wei, Finance-Specific Large Language Models: Advancing Sentiment Analysis and Return Prediction with Llama, 2.
[7] Cook, T. R., Kazinnik, S., Hansen, A. L., & McAdam, P. (2023). Evaluating local language models: An application to financial earnings calls. Available at SSRN 4627143. DOI: https://doi.org/10.2139/ssrn.4627143
[8] Dolphin, R., Dursun, J., Chow, J., Blankenship, J., Adams, K., & Pike, Q. (2024). Extracting Structured Insights from Financial News: An Augmented LLM Driven Approach. arXiv preprint arXiv:2407.15788.
[9] Drinkall, F., Pierrehumbert, J. B., & Zohren, S. (2024). Traditional Methods Outperform Generative LLMs at Forecasting Credit Ratings. arXiv preprint arXiv:2407.17624.
[10] Du, T., Kanodia, A., Brunborg, H., Vafa, K., & Athey, S. (2024). LABOR-LLM: Language-Based Occupational Representations with Large Language Models. arXiv preprint arXiv:2406.17972.
[11] Jansen, M., Swinkels, L., & Zhou, W. (2021). Anomalies in the China A-share market. Pacific-Basin Finance Journal, 68, 101607. DOI: https://doi.org/10.1016/j.pacfin.2021.101607
[12] Kim, A., Muhn, M., & Nikolaev, V. (2024). Financial statement analysis with large language models. arXiv preprint arXiv:2407.17866. DOI: https://doi.org/10.2139/ssrn.4835311
[13] Kim, S., Kim, S., Kim, Y., Park, J., Kim, S., Kim, M., ... & Lee, Y. (2023, November). LLMs Analyzing the Analysts: Do BERT and GPT Extract More Value from Financial Analyst Reports?. In Proceedings of the Fourth ACM International Conference on AI in Finance (pp. 383-391). DOI: https://doi.org/10.1145/3604237.3627721
[14] Kou, Z., Yu, H., Peng, J., & Chen, L. (2024). Automate Strategy Finding with LLM in Quant investment. arXiv preprint arXiv:2409.06289.
[15] Li, X., Li, Z., Shi, C., Xu, Y., Du, Q., Tan, M., ... & Lin, W. (2024). AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework. arXiv preprint arXiv:2403.12582.
[16] Ou, J. A., & Penman, S. H. (1989). Financial statement analysis and the prediction of stock returns. Journal of accounting and economics, 11(4), 295-329. DOI: https://doi.org/10.1016/0165-4101(89)90017-7
[17] Ouyang, K., Liu, Y., Li, S., Bao, R., Harimoto, K., & Sun, X. (2024). Modal-adaptive Knowledge-enhanced Graph-based Financial Prediction from Monetary Policy Conference Calls with LLM. arXiv preprint arXiv:2403.16055.
[18] Pan, X., Zhang, R., Li, K., Tang, L., & Zhang, H. (2019). A cross-sectional and comparative study on Chinese A-share and H-share stock markets. International Research Journal of Applied Finance, 10(2), 84-108.
[19] Srivastava, V. (2024). BAI-Arg LLM at the FinLLM Challenge Task: Earn While You Argue-Financial Argument Identification. In Proceedings of the Eighth Financial Technology and Natural Language Processing and the 1st Agent AI for Scenario Planning (pp. 165-173).
[20] Wang, Y., & Di Iorio, A. (2007). The cross section of expected stock returns in the Chinese A-share market. Global finance journal, 17(3), 335-349. DOI: https://doi.org/10.1016/j.gfj.2006.05.007
[21] Xing, F. (2024). Designing heterogeneous llm agents for financial sentiment analysis. ACM Transactions on Management Information Systems. DOI: https://doi.org/10.1145/3688399
[22] Zhao, H., Liu, Z., Wu, Z., Li, Y., Yang, T., Shu, P., ... & Liu, T. (2024). Revolutionizing finance with llms: An overview of applications and insights. arXiv preprint arXiv:2401.11641.
[23] Zhou, Y. (2024). Predicting CDS Spreads and Stock Returns with Weather Risk: A Study Utilizing Nlp/Llm and Ai Measures. Llm and Ai Measures. DOI: https://doi.org/10.2139/ssrn.4846844
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
