CECN1 Top-down Amplification
From Computational Cognitive Neuroscience Wiki
Contents |
Top-down Amplification
- The project file: amp_top_down.proj (click and Save As to download, then open in Emergent)
Back to CECN1 Projects
Project Documentation
(note: this is a literal copy from the simulation documentation -- it contains links that will not work within the wiki)
In the Network view panel you see a network with 3 layers: an input, and 2 bidirectionally connected hidden layers Hidden1 and Hidden2, with 1 unit in each layer.
- You can explore the weights using r.wt as before.
Now, we can pull up a graph view to plot the activations of the two hidden-layer units over time as the network settles in response to the input.
- Click the .T3Tab.CycleOutputData tab in the viewer at the right. Then in the .PanelTab.ControlPanel press Init and Run.
You should see something like figure 3.16 in the textbook in your graph window. The activation coming top-down from the Hidden2 unit is amplifying the relatively weak initial activation of the Hidden1 unit, resulting in the strong activation of both units. This is an excellent example of bootstrapping, because the Hidden1 unit has to activate the Hidden2 unit in the first place before it can receive the additional top-down excitation from it. Let's test the effects of the leak current.
Leak current
- Increase the strength of the leak current g_bar.l from 3.4 to 3.5 in the ControlPanel, and press Run. Next decrease the strength of the leak current and observe the effects.
You should see that with a higher leak current (3.5), the resulting Hidden1 activation is now insufficient to activate Hidden2, and no bootstrapping or amplification occurs. With decreases to the leak current, the bottom-up activation of Hidden1 is relatively strong, so that the bootstrapping and amplification from Hidden2 are less noticeable.
Word superiority effect
This simple case of bootstrapping and amplification provides some insight into the word-superiority effect studied by McClelland & Rumelhart (1981) and summarized in the introductory chapter. Recall that the basic effect is that people can recognize letters in the context of words better than letters in the context of nonwords. The puzzle is how word-level information can affect letter-level processing, when the letters presumably must be recognized before words can be activated in the first place. Based on this exploration, we can see how initially weak activation of letters (i.e., the Hidden 1 unit) can go up to the word-level representations (i.e., the Hidden 2 unit), and come back down to bootstrap and amplify corresponding letter-level representations.
Note that although this example is organized according to bottom-up and top-down processing, the same principles apply to lateral connectivity. Indeed, one could simply move the Hidden2 unit down into the same layer as Hidden1, where they would be just like two parts of one interconnected pattern, and if one were activated, it would be bootstrapped and amplified by the other.
