AI Colloquium: Arjen van Ooyen: Attention-Gated Reinforcement Learning

A central question that must be answered by any theory of category learning is how the brain learns to detect relevant sensory features while ignoring irrelevant ones. At the neuronal level this process corresponds to the tuning of a cell’s response profile, via synaptic modification, to become sensitive to the relevant features of a given category. Here we propose a novel learning theory called Attention-Gated Reinforcement Learning (AGREL), in which attentional feedback signals solve this fundamental problem. AGREL’s feedback connections are reciprocal to their feedforward counterparts, a property that allows the network to selectively target the lower level cells that initially drove the output. AGREL is designed in accordance with neurobiological principles and, in addition to feedback connections, employs a global reward signal to guide changes in connection strengths. This attentional gating of the reward signal offers a biologically realistic solution to the credit assignment problem, and in so doing provides a coherent and unifying framework for learning. We show that AGREL develops the same tuning curves as those found in a variety of behavioral primate categorization tasks. We further demonstrate the importance of AGREL’s two core elements: the feedback connections and reward signal. Following the removal of either element, the network was unable to develop the essential, and physiologically observed, internal representations required to correctly categorize the stimuli. These findings suggest that attentional feedback and reward signals are both necessary and sufficient to explain the formation of the category specific internal representations that have been observed to accompany learning.