Test Programs

From Emergent

How to make a set of test Programs for testing a network during or after training.

Wizard (in version 4.0.9 or higher)

With everything configured for training your network, you can do Programs/TestProgs in the Wizard, and it will create a set of algorithm-specific testing programs, and insert a Program Call to them in a Training program of your choosing, with options for when to call it. The information below shows how to do this manually, and is what the Wizard does.

Manual: Overview

1. Load in a set of programs for your algorithm (note: in 4.0.9 or higher, all algorithms have special testing programs, which allow you to skip the next 2 steps)
2. Delete the testing programs you don't want to use.
3. Add a command to set the network to TEST mode in the testing epoch program, and restore TRAIN mode after, and change the input_data to a var instead of an arg so you can use a different table for testing.
4. Update the ApplyInputs LayerWriter (see InputData for more details)
5. Add a call to the test program from your training program

Manual: Details

1. Go to programs group in project, right click, and New From Lib (note: do not select Load From Prog Lib, which will load over existing programs instead of creating new ones), and select the "All" version appropriate for your algorithm (e.g., BpAll_Std). NOTE: in 4.0.9 or higher, all algorithms now have a special xxAll_Test that saves you the next two steps (skip to step 4). This will load the programs into a new subgroup.

2. Usually, you just want to do testing at the level of the Epoch program, so you can select the Batch and Train level programs and delete them. It is also a good idea to rename these new programs with the word Test after each one.

3. Command to turn on and off TEST mode in the testing epoch program:

• do Edit Program in the Epoch program editor (see Program for details) and drag a memb= object from the Var/Fun toolbox on the left into the first line of the prog_code section of the epoch program. Set obj to the network variable, and click on member_lookup and select train_mode, which gets entered into the path. Then type in Network::TEST in the expr field.
• drag this new MemberAssign program element you just configured to the end of the program_code and drop it on top off the epoch_timer->EndTimer() line, and select Copy Here. Then, change the expr to be Network::TRAIN. This restores train mode when the testing is completed.
• near the top of the program, drag the input_data from args to vars, so that it won't be set by the calling program, allowing you to set a different input_data for testing than training.

4. Click on the ApplyInputs in the set of testing programs (not the one in training!) and do Edit Program, and click on the objs/LayerWriter object, and click the AutoConfig button. This should configure it the same as in the training program. If you've done something special in your apply inputs, you can always just drag the training one on top of your testing and select Assign To to copy it. Or, you can edit your Trial or Settle program to call the training one instead of the testing one.

5. Click on your training program where you want to call the testing. Usually this is the Train program. Do Edit Program.

• Decide when you want to do the testing:
  • If you want to test only at the end of training, drag a prog() object from the Var/Fun toolbox to the end of the program.
  • If you want to do it every epoch, drag it into the end of the loop_code for the WhileLoop that is the 2nd to last thing in the program.
  • If you want to do it only every few epochs, you can drag an if from the Ctrl toolbox and enter network->epoch % 10 == 0 or something like that as the expr -- this would be true every 10 epochs. You can make the 10 a variable to be even better! Then drag the prog() into the true code.
• Then click on the ProgramCall object (the prog()), and select the testing epoch program you created.