CCNBook/Sims/Motor/PVLV model details

From Computational Cognitive Neuroscience Wiki
Jump to: navigation, search

Detailed description of PVLV Layers


Figure PV.2: Complete PVLV model -- see text for details

The PVLV network is organized into several groups of layers based on neurobiological and/or computational function, including the various nuclei of the amygdala and different types of neurons in the ventral striatum, two areas that are involved in reward learning. Additionally, there are the PosPV and NegPV layers, which represent the rewards or punishments given to the network, as well as the VTA area, which releases dopamine, and the lateral habenula (LHb/RMTg) layer, which inhibits the dopamine layer.

Here is a description of what each layer does:

Input Layers

  • Stim_In -- This represents the particular stimuli (or CSes) in the environment encoded left to right ABCDEFUVWXYZ.
  • Context_In -- This represents conjunctive information about the particular CS that is being seen, and which context or location it is seen in. CSs are encoded in the depth (Y) dimension with three possible contexts for each CS in the horizontal dimension.
  • USTime_In -- This is a timing signal or clock specific to each CS and reward type, which increases as the CS is presented for a longer duration. The encoding scheme is somewhat complex due to the combinatorial requirement and includes the occurrence of specific USs themselves as possible predictors of the next occurrence as the bottom/foreground two unit groups. Positive valence is the left column of unit groups; negative is the right. Above/behind the two "pure US" cases are the CS-predictive cases with each CS mapping to a unit group. Within each unit group four distinct USs are in the depth dimension; time is on the horizontal. This scheme provides a highly differentiated signal to the network that allows it to learn which times rewards are expected relative to specific predictive events and USs. We make use of this abstract timing signal to capture the idea that prediction errors need to be temporally specific -- the brain representation of timing signals is likely to be more complex.
  • PosPV -- This represents four different kinds of rewards.
  • NegPV -- This represents four different kinds of punishments.

Amygdala Layers

These layers encode associations of particular CSes with rewards or punishments. The amygdala is an area of the brain that is important for making associations of different predictive cues with primary affective outcomes (such as different types or rewards or negative valence things like stress or pain). It is generally considered one of the most important regions of the limbic system, or brain's emotional centers.

  • BLAmygPosD1 -- This receives input from the LatAmyg, and ends up representing how strongly that specific stimulus in the LatAmyg has been paired with a specific type of reward. Each type of reward has it's own BAAcq unit.
  • BLAmygPosD2 -- This is an "extinction" neuron for positive rewards, and represents how likely the specific stimulus has been associated with the omission of reward.
  • BLAmygNegPosD2 -- This receives input from the LatAmyg, and ends up representing how strongly that specific stimulus in the LatAmyg has been paired with a specific type of punishment. Each type of punishment has it's own BAAcq unit.
  • BLAmygNegD1 -- This is an "extinction" neuron for punishments, and represents how likely the specific stimulus has been associated with omission of punishments.
  • CElAcqPosD1 -- This represents positive valence acquisition-coding cells of the central amygdala, lateral segment. It integrates information directly from Stimuli_In and from the BLAmygPosD1 to associate CSs with various rewards; sends excitatory projection to CEmPos.
  • CElExtPosD2 -- This represents positive valence extinction-coding cells of the central amygdala, lateral segment. It receives only from the BLAmygPosD2, which associates CSs with various omitted rewards; sends inhibitory projections to CElAcqPosD1 and directly to CEmPos.
  • CElAcqNegD2 -- This represents negative valence acquisition-coding cells of the central amygdala, lateral segment. It integrates information directly from Stimuli_In and from the BLAmygNegD2 to associate CSs with various punishments; sends excitatory projection to CEmNeg.
  • CElExtNegD1 -- This represents negative valence extinction-coding cells of the central amygdala, lateral segment. It receives only from the BLAmygNegD1, which associates CSs with various omitted punishments; sends inhibitory projections to CElAcqNegD2 and directly to CEmNeg.
  • CEmPos -- This integrates excitatory signals from CElAcqPosD1 and inhibitory signals from CElExtPosD2 to drive the dopamine response for a positive CS
  • CEmNeg -- This integrates excitatory signals from CElAcqNegD2 and inhibitory signals from CElExtNegD1, but does NOT drive dopamine signaling at all.


Ventral Striatum

The ventral striatum is particularly important for the associations of different cues with positive valence rewards, and generally shows inhibition for a negative valence outcome such as a punishment. It is part of the basal ganglia, which includes the dorsal (specialized for actions) and ventral striatum (specialized for reward information). It generally encodes a "value" signal, incorporating different costs such as effort, possibility of pain, and risk into an estimate of how good something is. Additionally, by projecting to the dorsal striatum, it can use these value signals to help take actions that lead to possible rewards. The PVLV model focuses on Pavlovian conditioning, which does not require taking actions to get outcomes, but influences the brain's dopamine signals which play an important role in action selection and selection of prefrontal cortical representations. The Patch layers in the network correspond to a particular type of MSNs (the main neurons in the ventral striatum), and encode the level of reward or punishment expectation at the specific time an outcome is expected. This allows them to drive dopamine bursts if a reward is better than expected, or a dopamine dip if it is worse than expected, by projecting to the lateral habenula, which projects to the VTA layer. The "D1" and "D2" correspond to MSNs with D1 and D2 receptors, which have been found to be specialized for anticipation of rewards or punishments, respectively.

  • VSPatchPosD1 -- This layer codes the expectation of reward, and receives a timing signal from the USTime_In layer to let it know what time reward is expected.
  • VSPatchNegD2 -- This layer codes the expectation of punishment, and receives a timing signal from the USTime_In layer to let it know what time reward is expected.
  • VSPatchPosD2 -- This layer codes the expectation of a reward omission (not fun!), and receives a timing signal from the USTime_In layer.
  • VSPatchNegD1 -- This layer codes the expectation of a punishment omission (which is a good thing), and receives a timing signal from the USTime_In layer. expected.

The Matrix layers in the network correspond to another type of MSNs in the ventral striatum. They encode the associations of particular CSes with rewards or punishments, and provide an additional signal to the amygdala layers that can cause bursts for positive stimuli and drive dips for negative stimuli. In particular, these layers are crucial for driving dopamine dips to negative stimuli, as there is no evidence that the CA_Neg layers would project to the lateral habenula to drive a dip (but there is some evidence suggesting that the striatum does). (Hikosaka 2015)

  • VSMatrixPosD1 -- This layer codes the expectation of a reward, and receives from the Stimuli_In layer which encodes particular CSs.
  • VSMatrixPosD2 -- This layer codes the expectation of not being rewarded, and receives from the Stimuli_In layer which encodes particular CSs.
  • VSMatrixNegD2 -- This layer codes the expectation of a punishment, and receives from the Stimuli_In layer which encodes particular CSs.
  • VSMatrixNegD1 -- This layer codes the expectation of not being punished, and receives from the Stimuli_In layer which encodes particular CSs.

Dopamine and habenula layers

Dopamine is the neurotransmitter in the brain that is released when positive outcomes occur (and it decreases for negative outcomes). Dopamine levels reflect the probability and amount of reward that is received, and, once a reward is predicted, move from the time of a reward to a predictive CS (or cue) that tells you that the reward is likely to occur.

  • PPTg -- This receives from CAPos and projects to the VTAp, which allows it to drive a dopamine burst depending on how good the CS is.
  • VTAp -- This area (VTA) releases dopamine, and integrates the info it is receiving from the CAPos/PPTg layers to drive dopamine bursts, with information from the lateral habenula, which inhibits it to cause a dopamine dip.
  • LHbRMTg -- This represents the lateral habenula, which integrates the information from the VSPatch and VSMatrix layers with the outcomes received in the PosPV or NegPV layers, to figure out the difference between the amount of reward or punishment that was expected and what actually occurred. For example, if a reward was expected but not received, the habenula gets excited, which inhibits the VTAp to cause a dopamine dip.
  • VTAn -- This part of the VTA represents some VTA neurons that show dopamine bursts for punishments (Brischoux 09), a atypical population that has been less well studied that the VTAp type of dopamine neurons.