Asymmetric Learning Rates for Positive and Negative Feedback: A Formal Model Comparison

Author/​Artist
McDonald, Kelsey [Browse]
Format
Senior thesis
Language
English
Description
63 pages

Details

Advisor(s)
Niv, Yael [Browse]
Contributor(s)
Norman, Ken [Browse]
Department
Princeton University. Department of Psychology [Browse]
Class year
2015
Summary note
Reinforcement learning studies how learning systems interact with their environments in order to maximize a numerical reward signal. One of the central concepts in reinforcement learning is the reward “prediction error”, which is the numerical value of the reward received minus the expected reward value. The classic Rescorla-Wagner model posits that the learning agent updates their original reward estimate by stepwise error-correction: multiplying the prediction error by a learning rate parameter. The main limitation with the Rescorla-Wagner model is the implied valence symmetry with which feedback updates an action’s value estimate. This contradicts evidence that learning from positive and negative feedback has different effects on behavior and decision-making. In this thesis, I conduct a formal model comparison of the Rescorla-Wagner model with an alternative class of asymmetric learning models which discriminate based on valence. Our analyses show that behavioral choice data in a probabilistic learning experiment is more accurately described by an asymmetric learning algorithm rather than a symmetric learning rule which does not discriminate based on valence.

Supplementary Information