Deep Convolutional Neural Networks

From emergent
Jump to: navigation, search

As of Version 8.0, Backpropagation (see Backprop 8.0 Update) contains support for main features of Deep Convolutional Neural Networks (DCNN's) which have been widely used in a number of different pattern recognition applications (e.g., the now-classic AlexNet model: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf )

  • The "dropout" regularizer has been added -- see this paper for full details: http://jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf
  • There is a MAX_POOL act_fun option which computes the max over all of its recv connections, and only backpropagates error back to that max connection.
  • Connection weights can be shared across units, to support convolutional networks. This is supported in the TiledGpRFPrjnSpec ProjectionSpec which creates appropriate overlapping tiled connectivity patterns -- click the share_cons flag.

See our CCN repository here: https://grey.colorado.edu/CompCogNeuro/index.php/CCN_Repository for downloadable DCNN models.