Header menu link for other important links
X
ACE (Actor-Critic-Explorer) paradigm for reinforcement learning in basal ganglia: Highlighting the role of subthalamic and pallidal nuclei
V. Srinivasa Chakravarthy
Published in
2010
Volume: 74
   
Issue: 1-3
Pages: 205 - 218
Abstract
We present a comprehensive model of basal ganglia in which the three important reinforcement learning components-Actor, Critic and Explorer (ACE),-are represented and their anatomical substrates are identified. Particularly, we identify the subthalamic-nucleus and globus pallidus externa (STN-GPe) loop as the Explorer, and argue that complex activity of STN and GPe neurons, found in experimental studies, provides the stochastic drive necessary for exploration. Simulations involving a two-link arm model show task-dependent variations in complexity of STN-GPe activity when the ACE network is trained to perform simple reaching movements. Complexity and average levels of STN-GPe activity are observed to be higher before training than in post-training conditions. Further, in order to simulate Parkinsonian conditions, when dopamine levels in substantia nigra portion of the model are reduced, the arm displayed, as a primary change, small amplitude movements, which on persistent network training, amplified to large amplitude unregulated movements reminiscent of Parkinsonian tremor. © 2010 Elsevier B.V.
About the journal
JournalNeurocomputing
ISSN09252312
Open AccessNo
Concepts (36)
  •  related image
    Actor critic
  •  related image
    BASAL GANGLIA
  •  related image
    COMPLEX ACTIVITY
  •  related image
    COMPREHENSIVE MODEL
  •  related image
    Dopamine
  •  related image
    Experimental studies
  •  related image
    GLOBUS PALLIDUS
  •  related image
    Large amplitude
  •  related image
    Network training
  •  related image
    Norepinephrine
  •  related image
    PARKINSONIAN TREMOR
  •  related image
    REACHING
  •  related image
    REACHING MOVEMENTS
  •  related image
    Small amplitude
  •  related image
    SUBSTANTIA NIGRA
  •  related image
    TRAINING CONDITIONS
  •  related image
    TWO-LINK
  •  related image
    Brain
  •  related image
    Reinforcement learning
  •  related image
    Computer simulation
  •  related image
    Article
  •  related image
    BASAL GANGLION
  •  related image
    DISEASE SIMULATION
  •  related image
    GLOBUS PALLIDUS
  •  related image
    Mathematical model
  •  related image
    MENTAL TASK
  •  related image
    Movement (physiology)
  •  related image
    Nerve cell
  •  related image
    Parkinsonism
  •  related image
    Priority journal
  •  related image
    Reinforcement
  •  related image
    Statistics
  •  related image
    SUBSTANTIA NIGRA
  •  related image
    SUBTHALAMIC NUCLEUS
  •  related image
    Tremor
  •  related image
    Upregulation