This page contains various user's experiences for setting parameters in the Leabra algorithm.
See Modeling Principles for some more general tips on how to go about modeling.
Randy's General Tips for Basic Networks
In a basic 3 layer Leabra network, the key parameters that make the biggest difference are the inhibition parameters and Hebbian learning:
- inhibition: Hidden layers should generally use either KWTA_AVG_INHIB or KWTA_KV2K with kwta_pt set to .6 in the former and .25 in the latter. Output layers often benefit somewhat by using KWTA_INHIB with kwta_pt set to .25, because they don't need the extra flexibility that these others provide. The main thing to manipulate is the kwta.pct parameter in the hidden layer(s), which generally can be between .10 and .25. With more systematic mappings where similarity-based generalization is beneficial, higher values are better. When the mappings are more random, sparser values are better, and sometimes even lower than .1 can work well (e.g., in the hippocampus model, for an extreme example).
- Hebbian: try the lmix.hebb value in .01, .001, .0001, 0.0, kinds of values, and then hone in on more detailed values around the one that works the best. Also the savg_cor.cor value can be important -- it defaults to .4, but .8 and 1.0 can also work better in some circumstances.
Tips for Updating Projects (and XCAL parameters)
If you have any projects that have set the following parameters on the LeabraNetwork to non-standard values, you should reset them and save with latest version:
- sse_unit_avg un-check
- sse_sqrt un-check
- cycle_max = 60 <-- XCAL uses ct_time parameters, which then (re)set cycle_max, so setting this directly has no effect!
- maxda_stopcrit = -1 <-- XCAL automatically (should) set maxda_stopcrit to -1 -- it doesn't use it -- every trial is same length.
- trg_max_act_stopcrit = 1
- norew_val = 0.5 <-- this is only in in PFC_BG_Layers/PVeLayer
- off_errs check
- on_errs check
The new XCAL default parameters are shown in the defaults for the LeabraUnitSpec act_avg field -- because both the old and new versions are supported, it might be a bit ambiguous, but here's the story for the new l_sq mode:
- l_sq = true -- this is what puts it into the new squared mode..
- l_gain = 60
- thr_min = 0.01
- thr_max = 0.9
- l_dt = 0.05
- ml_dt = 1.0
- NOTE: The remaining parameters in the LeabraUnitSpec act_avg field have not changed (not relevant for standard trial-based XCAL anyway -- only for fully continuous XCAL_C)
- NOTE: For XCAL, if the connection specification (in the LeabraConSpec) is set to wt_sig.off >= 1.1 the network may train poorly, and is never really any better, so use wt_sig.off = 1.0 as the standard offset for the sigmoidal weight function. The auto default will be updated but existing projects will have whatever they were originally set to.
- NOTE: Experimental results from MH, for self-organizing (Hebbian) set xcal.thr_l_mix=1, and lower the lrate by maybe half.
For general information about the XCAL learning model, please see the CECN textbook chapter on Learning.
void LeabraUnitSpec::BioParams ( bool gelin = true, float norm_sec = 0.001f, float norm_volt = 0.1f, float volt_off = -0.1f, float norm_amp = 1.0e-8f, float C_pF = 281.0f, float gbar_l_nS = 10.0f, float gbar_e_nS = 100.0f, float gbar_i_nS = 100.0f, float erev_l_mV = -70.0f, float erev_e_mV = 0.0f, float erev_i_mV = -75.0f, float act_thr_mV = -50.0f, float spk_thr_mV = 20.0f, float exp_slope_mV = 2.0f, float adapt_dt_time_ms = 144.0f, float adapt_vm_gain_nS = 4.0f, float adapt_spk_gain_nA = 0.0805 )
set parameters based on biologically-based values, using normalization scaling to convert into typical Leabra standard parameters. gelin = configure for gelin rate-code activations instead of discrete spiking (dt = 0.3, gain = 80, gelin flags on), norm_x are normalization values to convert from SI units to normalized values (defaults are 1ms = .001 s, 100mV with -100 mV offset to bring into 0-1 range between -100..0 mV, 1e-8 amps (makes g_bar, C, etc params nice). other defaults are based on the AdEx model of Brette & Gurstner (2005), which the SPIKE mode implements exactly with these default parameters -- last bit of name indicates the units in which this value must be provided (mV = millivolts, ms = milliseconds, pF = picofarads, nS = nanosiemens, nA = nanoamps)