Evaluating The Effectiveness of The Llama2 Large Language Model in Analyzing Economic Texts: A Comparative Analysis Using Diverse Data Sources

Zhengqi Han

doi:10.54097/jx198j77

Authors

Zhengqi Han

DOI:

https://doi.org/10.54097/jx198j77

Keywords:

llama2, Economic Text Analysis, Sentiment Analysis, Entity Recognition, Topic Modeling.

Abstract

This study investigates the effectiveness of the open-source llama2 large language model in analyzing various types of economic texts. We employ a comparative analysis approach, utilizing data from four diverse sources: Federal Reserve Economic Data (FRED), Edgar (EDGAR) Database from the U.S. Securities and Exchange Commission (SEC), International Monetary Fund (IMF) Data, and World Bank Open Data. We focus on the performance of llama2 for specific tasks like sentiment analysis, entity recognition, and topic modeling. The findings will contribute to understanding the potential and limitations of using large language models for extracting insights from diverse economic data sources.

Downloads

Download data is not yet available.

References

H. Touvron, L. Martin, and K. Stone, “Llama 2: Open Foundation and Fine-Tuned Chat Models”.

S. R. Baker, N. Bloom, and S. J. Davis, “Measuring Economic Policy Uncertainty*,” Q. J. Econ., vol. 131, no. 4, pp. 1593–1636, Nov. 2016, doi: 10/f9gb2x.

Hoberg G. and Phillips G. M., “Text-Based Network Industries and Endogenous Product Differentiation.” Rochester, NY, Jul. 03, 2015. doi: 10.2139/ssrn.1520062.

V. H. Larsen and L. A. Thorsrud, “The value of news for economic developments,” J. Econom., vol. 210, no. 1, pp. 203–218, May 2019, doi: 10/gf5nd5.

S. L. Shapiro, J. A. Astin, S. R. Bishop, and M. Cordova, “Mindfulness-Based Stress Reduction for Health Care Professionals: Results from a Randomized Trial.,” Int. J. Stress Manag., vol. 12, no. 2, pp. 164–176, May 2005, doi: 10/c8wzz4.

M. Z. Frank and W. Antweiler, “Is All that Talk Just Noise? The Information Content of Internet Stock Message Boards.” Rochester, NY, Aug. 21, 2001. doi: 10.2139/ssrn.282320.

P. C. Tetlock, “Giving Content to Investor Sentiment: The Role of Media in the Stock Market,” J. Finance, vol. 62, no. 3, pp. 1139–1168, 2007, doi: 10/bpzg7m.

N. Jegadeesh and D. (Andrew) Wu, “Word Power: A New Approach for Content Analysis.” Rochester, NY, Jul. 01, 2013. doi: 10.2139/ssrn.1787273.

K. Miwa, “Language barriers in analyst reports,” Int. Rev. Econ. Finance, vol. 75, pp. 223–236, Sep. 2021, doi: 10/gtkzjj.

“Business Cycle Narratives by Vegard H. Larsen, Leif Anders Thorsrud: SSRN.” Accessed: Mar. 07, 2024. [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3130108

R. Gruetzemacher, “The Power of Natural Language Processing,” Harvard Business Review, Apr. 19, 2022. Accessed: Mar. 07, 2024. [Online]. Available: https://hbr.org/2022/04/the-power-of-natural-language-processing

Y. Li, Y. Guo, F. Guerin, and C. Lin, “Evaluating Large Language Models for Generalization and Robustness via Data Compression.” arXiv, Feb. 03, 2024. doi: 10.48550/arXiv.2402.00861.