CECN1 Distributed Top-down Amplification

From Computational Cognitive Neuroscience Wiki

Jump to: navigation, search

Top-Down Amplification in a Distributed Network

Back to CECN1 Projects

Project Documentation

(note: this is a literal copy from the simulation documentation -- it contains links that will not work within the wiki)

  • To start, it is usually a good idea to do Object/Edit Dialog in

the menu just above this text, which will open this documentation in a separate window that you can more easily come back to. Alternatively, you can always return by clicking on the ProjectDocs tab at the top of this middle panel. The network here is like that in the previous example, except that now there are multiple units per layer.

  • Click on the .PanelTab.AmpTopDownDistNet tab to bring up the network view panel, and select r.wt as the variable to view, and then click on the three Hidden1 units with the red-arrow select tool.

Notice that they each receive one corresponding input from the Input units (this is called one-to-one connectivity). Notice also that the left and right hidden1 units receive uniquely from the left and right Hidden2 units, while the center Hidden1 unit receives from both hidden2> units.

  • Now click on the the left and right Hidden2 units.

Observe that the connectivity is symmetric, so that the left unit receives from the left and center Hidden1 units, while the right one receives from the center and right Hidden1 units.

Thus, the connectivity pattern can be interpreted as representing 3 separable features in the Input and Hidden1 units, with the Hidden2 units representing 2 "objects" each consisting of 2 out of these 3 features. As labeled in the simulation, you can think of the first object as a TV, which has the two features of a CRT and speakers, while the other is a synthesizer, which has speakers and a keyboard. Thus, there is a one-feature overlap between these objects, and it is this shared feature that will cause the network trouble.

Now we will present activity to only the left input unit, which is unique to the TV object, and observe the network's response. To see the trajectory of settling in the network, we can open a grid view.

As you can see by watching the network settle, and by looking at a trace of it in the grid log to the right (showing the activations for the Input, Hidden1, and Hidden2 units every 10 updates (cycles) of settling, the CRT hidden unit (on the left) first activates the TV unit, and then this comes back down to activate the Speakers feature. This is a good example of a pattern completion-like phenomenon that uses top-down activation instead of lateral activation. However, once the Speakers unit becomes activated, it then activates the Synth unit in the Hidden2 layer, which then does the same kind of top-down activation of the Keyboard unit. The result is the uninterpretable activation of all 3 hidden units.

Question 3.9 (a) List the values of g_bar.l where the network's behavior exhibited a qualitative transition in what was activated at the end of settling, and describe these network states. (b) Using the value of g_bar.l that activated only the desired two hidden units at the end of settling, try increasing the dt_vm parameter from .03 to .04, which will cause the network to settle faster in the same number of cycles by increasing the rate at which the membrane potential is updated on each cycle -- are you still able to activate only the left two hidden feature units? What does this tell you about your previous results?

Ambiguous stimuli

  • Next try to activate the ambiguous (center) input feature, Speakers, by assigning input_data in .PanelTab.ControlPanel to AmbigEnv. Init, and Run

One reasonable response of the network to this input would be to weakly activate the other features associated with this ambiguous input in Hidden1, indicating that it cannot choose between these two possibilities. This is impossible to achieve, however, because of the spreading activation phenomenon.

  • Run with this environment first with a leak value of 1.746, and then with a leak value of 1.745.

You can see that the network does not activate the other feature units at all with a leak of 1.746, whereas a value of 1.745 causes all of the units in the network to become strongly activated. The network exhibits strongly bimodal behavior, and with only a constant leak current to control the excitation, does not allow for graded levels of activation that would otherwise communicate useful information about things like ambiguity.

  • Next, set the leak current to 1.79 and assign input_data in the .PanelTab.ControlPanel to FullEnv, and then Init, and Run. This activates both the CRT and Speakers inputs. You should see that the activation overflows to the third feature unit.
  • Finally, increase the leak (g_bar.l) from 1.79 to 1.8 and Run.

Two inputs get weakly activated, but the TV unit in the Hidden2 layer does not. Thus, even with complete and unambiguous input for the TV features, activation either spreads unacceptably, or the network fails to get appropriately activated.

You should have observed from these explorations that bidirectional excitatory connectivity is a double-edged sword; although it can do some interesting amplification and pattern completion processing, it can also easily get carried away. In short, this type of connectivity acts like a microphone next to the speaker that it is driving (or a video camera pointed at its own monitor output) -- you get too much positive feedback.

It is useful to note that these bidirectional networks tend to be strongly bimodal and nonlinear with respect to small parameter changes (i.e., they either get activated or not, with little grey area in between). This is an important property of such networks -- one that will have implications in later chapters. This bimodal nonlinear network behavior is supported (and encouraged) by the nonlinearities present in the point neuron activation function (see section 2.5.4). In particular, the saturating nonlinearity property of the sigmoidal noisy X-over-X-plus-1 function provides a necessary upper limit to the positive feedback loop. Also important is the effect of the gain parameter "gamma", which magnifies changes around the threshold value and contributes to the all-or-nothing character of these units.

To continue on to the next simulation, close this project first (from the file menu). Or, if you wish to stop now, quit by selecting Quit Emergent from the main emergent menu.

Retrieved from "http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Distributed_Top-down_Amplification"
Personal tools