Build your own network
This tutorial shows you how to quickly setup your own standard Leabra network, and will introduce you to the key elements of an emergent project. This tutorial covers using the new project templates, which were introduced in emergent 5.3.0. While creating your first project, if you get lost at some point, don't worry. You can optionally close the project without saving, and start all over again. It won't take long to understand the basics of project creation, and you will soon be able to quickly create a new Leabra network.
Create A New Project
- Start by opening emergent and creating a standard Leabra project:
You can do this by selecting a "LeabraStd" template from the left panel and clicking OK or using the File/New Project menu (select LeabraStd to create a standard Leabra project, then click Ok).
- The new project will be created, and an emergent project window is opened.
Touring the Project
Your new project will open with 4 main panels (from left to right): programming toolbar, navigator, editor, and visualizer. The programming toolbar can be hidden/shown from the main menu, under View → Toolbars. The navigator, editor, and visualizer can be hidden/shown in any combination using the main menu View menu. For now, simply leave all 4 panels visible.
Let's start by browsing through the project structure (you may need to expand the navigator tree to see some of these objects). The navigator view contains the following major catagories for each project: docs, wizards, ctrl_panels, data, programs, viewers, and networks. We will be looking at a few of these catagories in more detail:
- From the top of the navigator view, select
Project_0and the main project properties will appear in the editor view. The project can be renamed here if you wish.
- In the navigator view, under wizards, select
LeabraWizard_0and the emergent wizard will appear in the editor view. If you had created your new project using the LeabraBlank template rather than the LeabraStd template, you would likely use this emergent wizard to create your network. We will not be using this wizard now, but it is worth reading through. Your new project was created as a Standard Everything. Also, take note of the Network, Data, and Programs buttons at the bottom of the LeabraWizard_0 control panel. These buttons can be used to manually setup a project.
- In the navigator view, under data, click through the default datatables:
EpochOutputData. Presently, these datatables are empty. In a few moments we will be updating StdInputData manually to provide network training data. The
EpochOutputDatawill contain the network performance monitoring data that will be recorded when the network is trained or tested.
- In the navigator view, under programs, check out each program under the LeabraAll_Std folder. This group of default programs will be used to train/test the network. As you click on each program, you can read a brief program description in the editor view.
- In the navigator view, under networks, select
Network_0and the network properties will appear in the editor view. The network properties can be customized, but for now we will leave them in their default state.
- In the navigator view, under the networks →
specsand you will see a summary list of the specifications utilized by the network. If you click through each specification in the tree view, then you can see a full listing of the default parameters for each specification type (layer spec, unit spec, bias spec, projection spec, and connection spec).
- In the navigator view, under the networks →
layersand you will see a summary list of the layers within the network. The defaults are set to generate a standard three layer network, with one Input layer, one Hidden layer, and one Output layer. The default geometry for each layer is 5 units x 5 units, and corresponds to the size of the data contained in the
- NOTE: Check the navigator; you should only have one network. If you accidentally created any extra networks you can right-click on the name for the extra network in the tree frame, and select the delete option to remove the extra networks.
- In the navigator view, under the networks →
- editor view tabs control TIP : If you observe the left-most tab in the editor view, you will note that the focus of the tab will automatically shift to whichever object is selected in the navigator view. You can return to the LeabraWizard_0 control panel in the editor view by selecting the object in the navigator view, or by selecting the tab in the main panel frame.
- pinned tabs TIP : The LeabraWizard_0 tab should remain available as it was automatically 'pinned', which means fixed in place. You can pin or unpin any tab by right-clicking on the tab itself and selecting 'Pin in Place' or 'Unpin'. If a tab is pinned it will not disappear from the tabs in the center panel when a new focus object is selected from the project tree. Remember that you can re-open any tab by simply selecting the corresponding object in the project tree.
Create Network Input and Output Patterns
StdInputData must be updated with training data in order for the network to learn anything. This data will be presented on the network's Input and Output layers to train the network to give particular output patterns in response to particular input patterns. For this demo, we will simply make a vertical line, a horizontal line, and a diagonal line as our inputs; with matching output patterns.
- In the navigator view, under the data section, click on
StdInputData. In the
StdInputDataeditor view, the Data Table button should be selected. The table does not contain any rows, but does contain the following columns:
- Name is a place to put a distinct name for each pattern you have created. The pattern name can be used in log files, or in programs.
- Input is a matrix-type column -- each Input matrix cell corresponds in dimension to the Input layer of your network. Each cell in the Input matrix will contain the value (from 0.0 to 1.0) that will get presented to the network during training and testing, in the Leabra 'plus' phase.
- Output is also a matrix-type column -- each Output matrix cell corresponds in dimensions to the Output layer of your network. Each cell will contain the value that you want your network to learn when it is presented with the Input pattern for that table entry (row).
- We will need to first add three rows to the datatable. In the
StdInputDataeditor view, select the Add Rows button. This will bring up an AddRows dialog box. Set
n=3, then hit Ok.
- Now we can enter our data. In the
StdInputDataeditor view, select the first matrix cell under the Input column. A matrix (table) will appear in the lower half of the panel, and you can manually type in values into each cell:
- For a vertical line: type a '1' into each cell down a single column in the Input (matrix) table
- For a horizontal line: type a '1' into each cell across a single row in the Input (matrix) table
- For a diagonal line: type a '1' into each cell diagonally across the Input (matrix) table
Because in this case we are going to train the network to output the same thing as the input, just enter the same values you entered for the Input matrix into the corresponding Output matrix. Alternatively, you can simply copy the matrix values by selecting a completed Input (matrix) by right-clicking on the (matrix) and choosing Copy, then right-click on the corresponding Output (matrix) and choosing Paste.
- Finally, you can also enter descriptive names for the rows in the 'Name' column (eg. vertical, horizontal, and diagonal). You should now have a completed StdInputData table, with names for each data row, and values entered for both the Input and Output data matrices in each data row. For example, your vertical data should look something like this:
NOTE: Check to make sure that both your input pattern and output pattern match for each pattern set which will be presented to the network (eg. vertical 'line' for vertical Input and Output patterns, horizontal 'line' for horizontal Input and Output patterns, diagonal 'line' for diagonal Input and Output patterns).
Run the Network
To see your network in action you will first need to train the network on the data you created, and then you can test the network to see how well it has learned your patterns.
- In the navigator, under programs, click on
LeabraTrain(you may need to first expand the tree by clicking on programs). This will cause a
LeabraTraincontrol panel to appear in the editor. The
LeabraTrainprogram will train your network given the
StdInputDatayou created. Take note of the Init, Run, Step, Stop, and Abort and other buttons at the bottom of the
- Click Init.
- Click Run -- your network should start running, and you should see it operating in the Visualizer panel.
During training the
StdInputData you created is presented to the network and the network weights are trained based on the Leabra algorithm. Depending on how fast your computer is, you should observe that each pattern is presented twice in succession:
- the first time is the "minus" phase, in which the network is allowed to run free -- since we started with initially random weights, the output will be random
- the second phase is the "plus" phase, in this phase, the network output is clamped to the desired value; the Leabra algorithm then compares the plus and minus phases, and applies a correction to the weights.
The LeabraTrain program calls LeabraEpoch, Epoch calls LeabraTrial, Trial calls LeabraQuarter (this is where "settling" occurs), Quarter calls LeabraCycle. As each plus and minus phase has a preset cycle count, if you simply initialize the network (Init), set Step: 10, and then press the Cycle button, the program will execute 10 cycles and will tend to transition into and output the plus and minus phases so you can observe the network learning. Watch the phase in the network text display in the Visualizer network view (right-hand panel). Over time, you should see the minus phase patterns starting to approximate the hard clamped output values (the pattern you created in your Output matrix in your StdInputData).
When the network has finished training, you can manually step through the patterns to see what the output is. To do this, you will change the default stepping program used by the network:
- You should still have the
LeabraTrainprogram selected in the tree and visible in the panel; if not, select it in the navigator, under programs, by clicking on
- In the
LeabraTraineditor, press the Program Ctrl button to show the control panel, and then set the train_mode=TEST
- At the bottom of the editor panel, set Step: 1, and then press the Quarter button. The program will execute 1 settling cycle.
Now you can press the Quarter button again to advance the network through another quarter of settling -- there are 25 cycles per quarter (3 quarters of minus phase and one of plus) the network will switch to the plus and minus phases with each step; in the plus phase you will see the exact output pattern displayed in the Output layer; in the minus phase, you should see a pattern of activation that is at least somewhat close to the output pattern (depending on how long your trained your network, and its batch termination criteria. (See Leabra for more information on the Leabra algorithm.)