Changes 8.0.0

From emergent
Revision as of 10:50, 8 May 2017 by Oreilly (Talk | contribs) (Major Changes)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Back to ChangeLog -- see emergent7 for ongoing support of the previous version (7). Svn Checkin Log for 8.0.0

Major Changes

  • Major changes to Network and associated objects To optimize multithreading performance (see Thread Optimization for details), memory is now allocated separately for each thread (CPU core), which required a complete rewrite of the way that all of the network algorithms are implemented. Here are a few highlights:
  • All the compute-relevant variables from Unit have been moved to a separate UnitVars data structure, which is what is allocated in thread-specific chunks, maintained by the Network object. To access e.g., the activation of the unit, you can call the act() method on the unit (which goes through the associated UnitVars), or you can call GetUnitVars() on the Unit and access the act member from there. The NetView and NetMonitor will automatically find these variables as before.
  • The connection groups (.recv and .send) have been removed from Unit and are also allocated separately as ConGroup structures (managed by the Network) -- the optimized algorithm computations now loop directly over these ConGroup objects, instead of going through the Unit level first. To access the connections from the Unit, you need to use the relevant methods on Unit, e.g., NRecvConGps() tells you how many receiving connection groups there are (was .recv.size before), and RecvConGroup(rgp_idx) returns a particular connection group indexed from 0..NRecvConGps().
  • bias connections have been eliminated as a separate Connection object, and replaced with individual bias_* members directly on the unit vars, e.g., the bias weight is now bias_wt in UnitVars.
  • Overall, you should think of the Layer and Unit objects as providing the structure template or "skeleton" of the network, and contain all the more static, gui-level information, whereas UnitVars contain the dynamic state variables that are updated through processing.
  • It is no longer possible to save or load the entire network at the Unit and connection level to a file -- you can only save the Layer-level structure of the network, which is then built dynamically when a Project is loaded. It is now possible to specify a weights file that is loaded automatically when the project or network is loaded -- this achieves most of the functionality of saving the network.
  • The USE_SSE8 compile option and configure script option have been removed -- vectorizing is now auto-detected based on processor type at compile time. See compile_info under Preferences, Expert items for full information on the compile of the executable, including the vectorization instruction set available (7 = AVX, 8 = AVX2 / haswell)
  • Network Building and weight initialization are optimized, with speed improvements of a factor of 10 or more! This is especially true for networks with symmetric weights (ConSpec wt_limits.sym).
  • Vectorizing has been improved, by ensuring that connections are chunked to allow vectorized access to the unit variables associated with the connections, and connections are allocated in one huge memory chunk, with proper alignment for vectorization. For Leabra, which uses sender-based connections, this means that the receiving units receiving those sending activations are aligned. The chunk size used for vectorization depends on the processor, and can be 4 or 8 (you can click on a SendCons inside the unit.send and see vec_chunked_size for how many connections were chunked, and under hidden items, vec_chunk_targ which is the target chunk size, now set to 8 (for compatibility across all platforms, where 8 is the maximum vector chunk size).
  • To keep up with the new vectorizing changes, the Mersenne Twister pseudo-random number generator (PRNG) (see MTRnd) has been updated with a version that is fully thread safe, with 100 different parameterizations of the PRNG that can be used by up to 100 different threads to generate distinct sequences of random numbers, which nevertheless are all cued off of the same initial seed, for full reproducibility. Also, this seed is just a single int, instead of the internal state of the PRNG. As a result, the system will automatically generate new random seeds in any of your RndSeed_List objects -- these are typically found in Train programs, and in the BatchRndSeed program in LeabraFlex programs. These changes mean that specific random sequences will not be preserved from pre-8.0 projects. But after this point, barring further changes, sequences should be consistent, even when using multiple threads.
  • GPU support, specifically NVIDIA's CUDA toolkit, has been implemented for Backpropagation, with 21x speedup relative to a single CPU (see GPU for details) -- the new Thread Optimization memory allocation enables most host-side memory structures to be transparently shared with the GPU device (with n_threads = 1 to keep the memory blocks coherent -- could extend to n_threads = number of GPU's). Very general framework now in place so much easier to support other algorithms -- Leabra is next (an earlier implementation exists but has not yet been updated to the new framework).
  • Major changes for the Leabra algorithm, described in Leabra_8.0_Update with instructions for how to update existing projects. The entire implementation was re-examined with the objective of removing rarely-used and legacy options, to make the edit dialogs and parameters dramatically simpler -- only the current, best-performing, most biologically-supported versions of algorithms are now supported.
  • Major changes to the Backpropagation implementation -- see Backprop_8.0_Update for instructions to update existing projects. All of the supported options are now available in the basic BpUnitSpec and BpConSpec, with enumerated options for things like activation functions etc. This is necessary for supporting the GPU implementation which does not support C++-style class hierarchies very well, and is also much easier for exploring different options without having to create different spec types.
  • Significant changes for keyboard shortcuts:
    • The meta key is no longer equivalent to the control key (on mac this means the command key is no longer equivalent to the control key). This opens the door to additional keyboard shortcuts/commands.
    • Key bindings can be set by the user. See the menu item "Tools->Choose Key Bindings"
    • To use the custom key bindings go to preferences and set "current key bindings" to "Key Bindings Custom"
    • The backspace and delete keys no longer delete objects (too easy to do accidentally). Use Control-D or the Delete menu item (or choose your own key combination!). To move an object use Cut followed by Paste
    • The preference "emacs_mode" is gone because you can choose your own keyboard shortcuts
  • Open Project from Web -- there is direct support in emergent for downloading and uploading Project and Program files to a properly configured wiki.
  • DataTable LoadData, LoadAnyData now have reset_first=true default arg, instead of false -- this means that the default is to overwrite any existing contents, instead of appending, which used to be the default -- loading over seems to be the more typical assumption, and forgetting to specify reset_first=true often results in repetitive unintended appending of data to a table, causing an effective memory leak. Any Program code that relied on the prior append default will need to be fixed to specify reset_true=false!!
  • Source code reorganized, breaking up the single src/temt/ta directory into separate directories, to make it easier to find things: ta_core, ta_math, ta_program (all the Program code), ta_data (all the DataTable related code), ta_proj (taProject and associated objects), ta_gui, ta_3d (3d gui elements)). Also, all the virtual environment and sensory processing stuff has been moved to src/emergent/virt_env

Minor Changes

  • search algorithms improved and simplified -- new syntax for specifying ranges of values, and simplified interface for writing search algorithms -- existing basic ones fully tested -- can be toggled on and off.
  • can now specify a given project name to use for the svn repository (in the Properties tab), instead of using the actual project name -- this allows multiple different project files to share a common svn repository, which is useful for trying out different architectural variants in different projects.
  • support for multiple users, and Update is MUCH faster when doing so
  • simplified flag for recording parameters vs. searching params
  • jobs_deleted table now accumulates all deleted jobs, including the svn number in last_svn needed to recover the job from svn -- the UnDelete function can now recover selected jobs -- facilitates keeping the jobs_done list, and the active svn list of files, smaller, while never losing anything really..
  • SaveState menu action triggers save of weights at any point, and system auto-saves weights 5 minutes prior to designated job end time.
  • Stampede super computer is available to emergent users to run large-scale models -- a great resource!
  • GridView fixes: Better performance with large data tables, set the paging size (i.e. set the number of rows/cols you jump when clicking << or >>)
  • hide/show labels on axes
  • new color mode "Color by Group" provides color coding and legend when comparing variables across groups, for example data from models run with different parameters.
  • each Y variable now has an edit button for setting line color, style, etc.
  • stop_step_cond condition expression for when they should obey the step (and stop) buttons. This allows more powerful stepping behavior, e.g., as implemented in the LeabraFlex programs MasterRun program, which can now step at the epoch level (and whenever there is an error, or a correct trial).
  • Can now directly call functions in other Programs -- functions must not take any args -- works just like calling another program (including setting the overall global args of the Program), except that it then just calls the named function. This allows Programs to function much like C++ classes, where the functions in the Program are now like methods on a class and can be called directly -- makes it easier to have one centralized Program that handles various aspects of a given functionality (e.g., everything to do with generating and updating a dynamic environment).
  • RunFunction button allows you to run program functions (that take no args) directly through the gui.
  • Conditional printing replaced with simpler mechanism that uses two booleans, one on the print statement and one on the program group. If the "debug" boolean on the print statement is false (default) then the print statement will execute but if true it will only print if the program group "debug_mode" boolean is also true. By toggling the program group debug_mode (at any level) you can turn printing on or off for all of the print statements that have debug set to true
  • Enhanced "lookup" - lookup now works for many cases not previously handled - use control key + l (that's a lowercase 'L')
  • Many fixes to parsing of typed ProgCode
  • support for Matrix access (i.e., a matrix on the left-hand-side of an = assignment) with a special ProgEl -- don't need to make these into css expressions anymore.
  • The menus/buttons "New From Program Lib" and "Load From Program Lib" are now "Add From Program Lib" and "Update From Program Lib" - hope that makes more sense
  • Console commands you type are now saved and reloaded when you relaunch emergent. Control+p or up-arrow to scroll back through commands.
  • Programming toolbar now has "new" objects you can drag into the navigator. For example, drag in a new data table, program, function, layer, etc
  • ReadOnly data table columns have a light grey shading to distinguish them and doubling clicking a readonly cell displays the cell content in a new window for easy viewing
  • Boolean-valued data table columns -- have a convenient check box toggle in the gui.
  • You can now add a Data Table cell to a control panel. Select a single cell and choose the context menu item "Add to Control Panel"
  • Two versions of searching the Navigator tree selectable by a pop up menu below the tree
  • "Find" which only looks at the visible text in the tree
  • "Deep Find" which looks at object members, descriptions, etc
  • Incremental search and replace - In addition to Find and Deep Find there is Replace which allows you to advance through the found items and replace or skip
  • Search widget added below data tables for searching scalar and matrix columns. Default is a "contains" string search with option for "exact match".
  • Better control of tree expansion state
  • Preferences for default expansion of top level groups (docs, ctrl_panels, data, programs, etc)
  • Preferences for default expansion of program subgroups (objs, args, types, vars, etc).
  • Separate defaults for programs in Navigator tree (left panel) and Program Editor (middle panel)
  • Also you can control whether "Call Arguments" are shown when you expand - default is don't show
  • Navigator tree state now saved with project. The tree will automatically expand to the saved state when the project is reopened
  • Menu item and shortcuts for expand collapse
  • Double-clicking a collapsed tree node will expand the node to its default; Double-clicking an expanded node will collapse it
  • Double-clicking while pressing the control key (command key on Mac) will expand the node fully
  • Layers now have both absolute and relative positions, and Network has an ABS_POS flag for selecting which one you want to be editable. Also, there is a new PositionLayersFromTable program in the Program Library that can grab layer positions and set them from values stored in a DataTable -- very handy for large, complex networks. Lastly, the MANUAL_POS flag on Network is now set on by default -- prevents automatic moving of layers when they overlap.
  • Unused specs are now shown with a grey background to indicate they are not used
  • The "Where Used" menu/button command on specs now lists child specs in use as well as the specific spec that was the target of the query.
  • New "Compare with Children" feature creates a data table showing overridden spec values. The table will be created in the group "spec_tables" under "networks". Makes it very easy to compare parent and child specs.
  • You can compare peer "specs" by choosing 2 or more and selecting the context menu item "Compare Peers". The table will be created in the group "spec_tables".
  • NetMonitor items can specify layers.varname to specifically target a variable on a layer, for all layers in the network -- this is the preferred way to do so, as it avoids confusion with network level objects having the same name. Also the Leabra standard monitors have been updated with many more useful items, and updated with new variable names -- recommended to update your monitors to match those in the LeabraEpochMonitor and LeabraTrialMonitor standard Program Library programs.
  • added new ParamSeq parameter sequencing capability, triggered off of the network epoch parameter, and using the same parameter access technology as the ParamSet -- can use this to define any kind of trajectory of any kind of parameter across training. MUST add call to network.Init_Epoch() method in appropriate place in the code (e.g., LeabraEpoch program or EpochStart in LeabraFlex programs -- standard program library has been updated with this call).
  • NetView allows you to select (in view control panel, check boxes) which variables appear in the "hot" list within the 3D network view panel (persists) -- different models have different relevant vars. Many fewer now appear there by default.
  • Other improvements:
  • Changes to color scale values are remembered so turning off auto_scale restores the last values.
  • New preference allows you to turn off Project level event logging (no .plog file will be created) -- this is only really needed for debugging. a general log is however always recorded in the user "library" emergent directory (e.g., ~/Library/Emergent/log on mac, ~/lib/Emergent/log on Unix)
  • New method SetMember() allows you to set a member value for multiple objects simultaneously. Very useful for setting a value on multiple "specs". To use select one or more items from the navigation tree. Children are automatically updated.
  • Font size preferences for specific parts of the GUI - labels, console, navigator and editor
  • If you are doing a demo try "View/Incr Font Size" to dynamically enlarge all fonts so your audience can read the text

Notable Bug Fixes

  • MANY Major periodic crashing bugs in the gui fixed, by preventing re-entry into the GUI event loop from delayed processing waitproc code -- things were disappearing in the middle of some of these operations.. software should be MUCH more stable now!
  • 1290 - some locale settings caused emergent to crash on windows and linux when opening a project -- no longer need to manually change your locale -- the program automatically uses the C locale which is compatible with the CSS programming language and expressions used throughout.
  • A bug in "Find" and "Deep Find" that was causing some items to be counted multiple times has been fixed

8.0.1 -- Minor Additional Release!

Svn Checkin Log for 8.0.1

  • New font size preferences and underlying code.
  • Update to PBWM learning algorithm and other parameters, and corresponding LeabraWizard.
  • Fixed a major issue that caused files to be overly large and Undo process to be overly slow: many optimizations for DataTable data saving were not being applied to the row_indexes of the data table, which can get quite large. Also improved undo debugging mechanisms.