Reinforcement Learning Inspired by Psychology and Neuroscience

Hongkun Wu

doi:10.54097/ehss.v8i.4673

Authors

Hongkun Wu

DOI:

https://doi.org/10.54097/ehss.v8i.4673

Keywords:

Reinforcement learning; cognition; reward; artificial intelligence.

Abstract

Decision-making is a crucial intelligence shared by animals and humans and it helps them to act in an intricate environment to seek rewards and avoid punishments. Psychologists first became interested in this ability and studied the conditional behavioral problems associated with it, then these studies have further led to the need for unified quantitative explanation models, among which Reinforcement learning is still the most convincing and data-backed model today. The model itself, in turn, facilitates research in neuroscience. In this paper, the researcher first introduces the original framework of reinforcement learning and the potential neural correlates to it. Then the paper reviews new developments in reinforcement learning algorithms that address the limitations of the original model as well as variants further inspired by neuroscience. Finally, the study highlights some new directions for future research. This study focuses on the evolution of reinforcement learning algorithms inspired by neuroscience, shows the relationship of mutual promotion and common development between reinforcement Learning and neuroscience, and clarifies some concerns for future exploration.

Downloads

Download data is not yet available.

References

Sutton R S, Barto A G. Reinforcement learning: An introduction[J]. Robotica, 1999, 17(2): 229-235..

Botvinick M M, Niv Y, Barto A G. Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective[J]. Cognition, 2009, 113(3): 262-280.

Yerkes R M, Morgulis S. The method of Pawlow in animal psychology[J]. Psychological Bulletin, 1909, 6(8): 257.

Sutton R S, Barto A G. Time-derivative models of pavlovian reinforcement[J]. 1990.

Redish A D, Jensen S, Johnson A, et al. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling[J]. Psychological review, 2007, 114(3): 784.

Silver D, Schrittwieser J, Simonyan K, et al. Mastering the game of go without human knowledge[J]. nature, 2017, 550(7676): 354-359.

Shalev-Shwartz S, Shammah S, Shashua A. Safe, multi-agent, reinforcement learning for autonomous driving[J]. arXiv preprint arXiv:1610.03295, 2016.

Fanselow M S. Pavlovian conditioning, negative feedback, and blocking: mechanisms that regulate association formation[J]. Neuron, 1998, 20(4): 625-627.

Dayan P, Abbott L F. Theoretical neuroscience: computational and mathematical modeling of neural systems[M]. MIT press, 2005.

Watkins C J C H. Learning from delayed rewards[J]. 1989.

Bertsekas D, Tsitsiklis J N. Neuro-dynamic programming[M]. Athena Scientific, 1996.

Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task[J]. Journal of neuroscience, 1993, 13(3): 900-913.

Montague P R, Dayan P, Sejnowski T J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning[J]. Journal of neuroscience, 1996, 16(5): 1936-1947.

Barto A G. 1 ‘1 Adaptive Critics and the Basal Ganglia,.’[J]. Models of information processing in the basal ganglia, 1995, 215.

Houk J C, Adams J L. 13 A model of how the basal ganglia generate and use neural signals that[J]. Models of information processing in the basal ganglia, 1995: 249.

Berns G S, McClure S M, Pagnoni G, et al. Predictability modulates human brain response to reward[J]. Journal of neuroscience, 2001, 21(8): 2793-2798.

Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning[J]. arXiv preprint arXiv:1312.5602, 2013.

Dayan P, Daw N D. Decision theory, reinforcement learning, and the brain[J]. Cognitive, Affective, & Behavioral Neuroscience, 2008, 8(4): 429-453.