emergent 8.2.2 BpUnitSpec Class Reference
emergent 8.2.2   Home · Wiki Docs For: BpUnitSpec · Emergent Help Browser 

BpUnitSpec Class Reference

Backpropagation version of unit spec

See for more info: Wiki Docs For: BpUnitSpec

 #include <BpUnitSpec>

defined at: /mnt/ssd/grey/local/include/Emergent/BpUnitSpec.h :41-63

Inherits From: UnitSpec, BaseSpec, taNBase, taOBase, taBase

Inherited By: BpUnitSpec*, BpUnitSpec&, const BpUnitSpec

Index: SubTypes, Members, Methods, Static Members, Static Methods, Expert Members, Expert Methods

Sub Types


SubType Documentation

enum BaseSpec::UsedStatus

ConstantValueDescription
BaseSpec::UNUSED0x00000000this spec has no children and is unused OR this spec and none of its children are used by any objects
BaseSpec::USED0x00000001this spec has no children and is used OR this spec and all of its children are used
BaseSpec::PARENT_USED0x00000002this spec is used but one or more of its child specs are unused
BaseSpec::CHILD_USED0x00000003this spec is not used but one or more of its child specs are used

enum BpUnitSpec::BpActFun

backprop activation function to use

ConstantValueDescription
BpUnitSpec::SIGMOID0x00000000standard sigmoidal activation in 0-1 range: 1 / (1 + e^-net) -- note that we dropped gain and offset params in 8.0 -- just uses standard gain = 1, 0 offset
BpUnitSpec::TANH0x00000001hyperbolic tangent function -- basically a sigmoid centered around 0 with range -1..1 (which is how we implement it)
BpUnitSpec::RELU0x00000002rectified linear unit, i.e., a simple threshold nonlinearity --- if netinput < 0, activation = 0, else act = net
BpUnitSpec::LINEAR0x00000003purely linear output -- not suitable for hidden units, as some form of nonlinearity is essential, but can be useful for output units
BpUnitSpec::NLXX10x00000004noisy-linear version of XX1 function = x/(x+1) (= 1/(1+x^1) -- sigmoidal asymptote but hard 0 threshold) where x = netin -- NL version adds a small linear 'ramp' for negative inputs starting at ramp_start negative input and reaching a magnitude of ramp_max at 0, after which point it transitions to regular XX1 (starting at ramp_max) -- ramp approximates effects of convolving XX1 with gaussian noise, but with a simpler function -- provides a hybrid of sigmoid and ReLu with good overall properties for sparse representations with bias set to -2
BpUnitSpec::BINARY0x00000005stochastic binary activation -- produces a 1 or a 0 based on sigmoidal underlying probability
BpUnitSpec::GAUSS0x00000006gaussian applied to the standard dot-product netinput -- also known as a 'bump' function
BpUnitSpec::RBF0x00000007radial basis function activation -- uses distance-based net input (net input is distance between activations and weights, instead of usual dot product) and runs that through a gaussian function to produce a radial basis function kernel activation
BpUnitSpec::MAX_POOL0x00000008compute the activation = max over input activations, and send backprop error only back to max input, rest = 0
BpUnitSpec::SOFTMAX0x00000009soft-max over the units within the layer -- does a few passes to compute activation -- useful for single-winner output layers (e.g., localist classification outputs) -- unlike in 8.0, this does NOT require an exponential layer input -- everything is done internally -- implies CROSS_ENTROPY error so that derivative is simple, and is ONLY applicable to output (TARGET) layers

enum BpUnitSpec::BpErrFun

type of error function to use -- replaces err_fun setting from earlier versions

ConstantValueDescription
BpUnitSpec::SQUARED_ERR0x00000000use the squared error function in computing derivatives of Target layer units -- appropriate for unbounded linear targets
BpUnitSpec::CROSS_ENTROPY0x00000001use the cross-entropy (sigmoid-based) error function in computing derivatives of Target layer units -- appropriate for binary targets

enum BpUnitSpec::BpNoiseType

ConstantValueDescription
BpUnitSpec::NO_NOISE0x00000000no noise
BpUnitSpec::ACT_NOISE0x00000001add noise to activations
BpUnitSpec::NETIN_NOISE0x00000002add noise to netinputs
BpUnitSpec::DROPOUT0x00000003drop out (zero activations) of units according to probability of noise.mean (like multiplying by a bernoulli distribution)


Regular (preferred) Member and Method Documentation

Members

Member Category: Activation

Member Category: Statistic

Member Category: Structure

Member Category: _NoCategory

Member Category: taBase

Methods

Method Category: Access

Method Category: Activation

Method Category: Learning

Method Category: Modify

Method Category: ObjectMgmt

Method Category: State

Method Category: Statistic

Method Category: Structure

Method Category: UserData

Method Category: _NoCategory


Member Documentation

Member Category: Activation

UnitSpec::act_range : MinMaxRange

range of activation for units

Member Category: Statistic

UnitSpec::sse_tol : float

tolerance for computing sum-squared error and other error metrics on a per-unit basis

Member Category: Structure

UnitSpec::bias_spec : ConSpec_SPtr

con spec that controls the bias connection on the unit

Member Category: _NoCategory

BpUnitSpec::act_fun : BpUnitSpec::BpActFun

activation function to use -- note that act_range is ignored for most functions except linear-based ones, and for output units using the cross-entropy error function

BaseSpec::desc : taString

Description of what this variable is for

BpUnitSpec::err_tol : float

error tolerance: no error signal for a unit if |targ-act| < err_tol) (i.e., as if act == targ exactly) -- often useful to set to .05 or so to prevent over-learning with binary training signals -- big weights often needed to get very high or low activations

BpUnitSpec::error_fun : BpUnitSpec::BpErrFun

error function to use: only applicable to layers of type TARGET -- squared error is appropriate for unbounded linear targets, while cross-entropy is more appropriate for binary targets

BpUnitSpec::gauss : GaussActSpec

specs for Gaussian bump or RBF activation function when that is being used

BpUnitSpec::nlxx1 : NLXX1ActSpec

specs for nlxx1 function when that is being used

BpUnitSpec::noise : Random

distribution parameters for random added noise

BpUnitSpec::noise_type : BpUnitSpec::BpNoiseType

type of noise to use

BpUnitSpec::save_err : bool

whether to save the actual error value -- this requires extra computation and is not necessary for learning -- just for instructional / informational purposes

Member Category: taBase

taNBase::name : taString

name of the object


Method Documentation

Method Category: Access

Variant BaseSpec::Elem ( Variant& idx, taBase::IndexMode mode = IDX_UNK )

get element(s) from container -- return can be a single item or a Matrix of multiple items, depending on the index -- see IndexMode for all the possible indexing modes and associated return values (some of which are not applicable to some containers, as documented for that container)

Show Source Code

Method Category: Activation

void UnitSpec::ApplyInputData ( UnitState_cpp* u, NetworkState_cpp* net, float val, UnitState_cpp::ExtFlags act_ext_flags, bool na_by_range )

apply input data value according to ext flags

Show Source Code

void BpUnitSpec::Compute_Act ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

compute the activation value of the unit: what it sends to other units

Show Source Code

void BpUnitSpec::Compute_Netin ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

compute net input: activations times weights (receiver based)

Show Source Code

void UnitSpec::Compute_NetinAct ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

compute the netinput (receiver-based) and then activation value of the unit: what it sends to other units

Show Source Code

void UnitSpec::Compute_SentNetin ( UnitState_cpp* u, NetworkState_cpp* net, float sent_netin )

called by network-level Send_Netin function to integrate sent netin value with current net input value -- default is just to set to net val + bias wt if avail

Show Source Code

void BpUnitSpec::Init_Acts ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

initialize unit activation state variables

Show Source Code

void UnitSpec::Init_InputData ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

initialize unit activation state variables for input data

Show Source Code

void UnitSpec::Init_UnitState ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

initialize all unit state -- do NOT access anything else, e.g., at congroup level

Show Source Code

Method Category: Learning

void BpUnitSpec::Compute_Weights ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

update weights from deltas

Show Source Code

void BpUnitSpec::Compute_dWt ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

compute change in weights: the mechanism of learning

Show Source Code

void UnitSpec::Init_Weights ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

init weight state variables

Show Source Code

void UnitSpec::Init_dWt ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

initialize the weight change variables

Show Source Code

Method Category: Modify

BaseSpec* BaseSpec::NewChild ( TypeDef* child_type )

create a new child spec that inherits from this one but can have its own unique parameters

Show Source Code

Method Category: ObjectMgmt

bool UnitSpec::CheckConfig_Unit ( Layer* lay, bool quiet = false )

check for for misc configuration settings required by different algorithms

Show Source Code

void BaseSpec::Defaults ( )

restore specs to their default original parameter values, for parameters that have a strong default value -- WARNING: you will lose any unique parameters for anything that has a strong default value

Show Source Code

taString taNBase::GetName ( )

Get the name of the object

Show Source Code

bool taNBase::HasName ( )

does the object have a name field that can be set?

Show Source Code

void taNBase::MakeNameUnique ( )

make sure my name is unique relative to names of objects associated with my owner (e.g., if it is a list object), typically because my name has changed, and owner needs to ensure that all names are unique

Show Source Code

void BaseSpec::SetMember ( taString& member, taString& value )

Set member to value for each of the selected objects. Member will be the member name or will be object_name.member_name, e.g. input_size.retina_size.x, replace any space with '_' . This also automatically sets the spec unique flag to 'On' so that this value is unique to this spec, and updates all the children of this spec so they can inherit this change (or not depending on their unique flags).

Show Source Code

bool taNBase::SetName ( taString& nm )

Set the object's name

Show Source Code

Method Category: State

ConSpec* UnitSpec::GetMainBiasSpec ( )

get the main bias spec (not cpp)

Show Source Code

int BpUnitSpec::GetStateSpecType ( )

derived classes MUST override this and pass correct global type id these methods keep sigmoidal-type values within the correct range to prevent numerical errors

Show Source Code

Method Category: Statistic

bool UnitSpec::Compute_PRerr ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no, float& true_pos, float& false_pos, float& false_neg, float& true_neg )

compute precision and recall error statistics for this unit -- true positive, false positive, false negative and true negative -- returns true if unit actually has a target value specified (otherwise everything is 0) -- precision = tp / (tp + fp), recall = tp / (tp + fn), fmeasure = 2 * p * r / (p + r), mcc = ((tp*tn) - (fp*fn)) / sqrt((tp+fp)*(tp+fn)*(tn+fp)*(tn+fn))

Show Source Code

float UnitSpec::Compute_SSE ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no, bool& has_targ )

compute sum squared error for this unit -- uses sse_tol so error is 0 if within tolerance -- has_targ indicates if there is actually a target value (else the return value is 0)

Show Source Code

Method Category: Structure

ConSpec_cpp* UnitSpec::GetBiasSpec ( NetworkState_cpp* net )

get the bias con spec

Show Source Code

void UnitSpec::Init_Weights_post ( UnitState_cpp* u, NetworkState_cpp* net, int thr_no )

post-initialize state variables (ie. for scaling symmetrical weights, other wt state keyed off of weights, etc) -- threaded for speed

Show Source Code

void UnitSpec::LoadBiasWtVal ( float bwt, UnitState_cpp* u, NetworkState_cpp* net )

load bias weight value into bias weights of given unit -- handles any post-loading updates directly

Show Source Code

Method Category: UserData

void taOBase::RemoveAllUserData ( )

get rid of our user data list entirely -- this is done automatically when saving something that has no user data items, but you can also force it with this method -- deletes the whole list

Show Source Code

Method Category: _NoCategory

float BpUnitSpec::ActDeriv ( float netin, float act, int thr_no )

compute derivative of activation from either netin or act

Show Source Code

float BpUnitSpec::ActFromNetin ( float netin, int thr_no )

compute activation from netinput

Show Source Code

bool taNBase::AddFromTemplate ( taBase* obj, bool& is_acceptable )

handles drops from toolbar - when adding an object to a program, network, etc - e.g. dropping generic data table onto a program - set is_acceptable for objects 'not handled' but which are acceptable

Show Source Code

void BaseSpec::ApplyTo ( SpecUser* obj1, SpecUser* obj2 = __null, SpecUser* obj3 = __null, SpecUser* obj4 = __null )

apply this spec to given object(s) (leave NULL any that are not needed)

Show Source Code

bool BaseSpec::CheckObjectType ( taBase* obj )

checks object type, issues error and returns false if not sufficient

Show Source Code

bool BaseSpec::CheckType ( TypeDef* td )

checks typedef type, issues error and returns false if not sufficient

Show Source Code

float BpUnitSpec::ClipSigAct ( float y )

Show Source Code

float BpUnitSpec::ClipSigNet ( float x )

different activation and error derivative functions

Show Source Code

float BpUnitSpec::ClipTanhAct ( float y )

Show Source Code

void BaseSpec::ComparePeers ( BaseSpec* spec )

calls network::SpecComparePeers to populate a table that compares this specs member values with its children's values - this data does not update - call again if you edit spec values!

Show Source Code

void BaseSpec::CompareWithChildren ( )

calls network::SpecCompareWithChildren to populate a table that compares this specs member values with its children's values - this data does not update - call again if you edit spec values!

Show Source Code

void BpUnitSpec::Compute_Error ( BpUnitState_cpp* u, BpNetworkState_cpp* net, int thr_no )

call the unit error function (only on target units)

Show Source Code

void BpUnitSpec::Compute_MaxPoolNetin ( BpUnitState_cpp* u, BpNetworkState_cpp* net, int thr_no )

compute netin = max over sending acts for max_pool case

Show Source Code

void BpUnitSpec::Compute_dEdA ( BpUnitState_cpp* u, BpNetworkState_cpp* net, int thr_no )

compute derivative of error with respect to unit activation

Show Source Code

void BpUnitSpec::Compute_dEdA_dEdNet ( BpUnitState_cpp* u, BpNetworkState_cpp* net, int thr_no )

compute both derivatives in sequence

Show Source Code

void BpUnitSpec::Compute_dEdNet ( BpUnitState_cpp* u, BpNetworkState_cpp* net, int thr_no )

compute derivative of error with respect to unit net input

Show Source Code

BaseSpec* BaseSpec::FindMakeChild ( taString& nm, TypeDef* td = __null, bool& nw_itm = nw_itm_def_arg, taString& alt_nm )

find a child spec of given name, and if not, make it (if nm is not found and alt_nm != NULL, it is searched for)

Show Source Code

BaseSpec* BaseSpec::FindParent ( )

Find the parent spec of this one

Show Source Code

taBase* taOBase::GetOwner ( )

Show Source Code

TypeDef* BpUnitSpec::GetTypeDef ( )

Show Source Code

bool BaseSpec::GetUnique ( taString& memb_nm )

check inherit bit

Show Source Code

void BpUnitSpec::GraphActFun ( DataTable* graph_data, float min = -5.0, float max = 5.0 )

graph the activation function along with derivative (NULL = new graph log)

Show Source Code

void UnitSpec::Initialize_core ( )

//////////////////////////////////////////////////////////////////////////////

Show Source Code

bool BaseSpec::IsInheritedAndHasParent ( taString& memb_nm )

is given member inherited from its parent (i.e., NOT unique) and parent actually has this member ot inherit from

Show Source Code

float BpUnitSpec::NLXX1Deriv ( float netin )

Show Source Code

float BpUnitSpec::NLXX1Fun ( float netin )

Show Source Code

float BpUnitSpec::ReLuDeriv ( float act )

Show Source Code

float BpUnitSpec::ReLuFun ( float netin )

Show Source Code

bool BaseSpec::RemoveChild ( taString& nm, TypeDef* td = __null )

remove a child based on name or type

Show Source Code

void BaseSpec::SetUnique ( taString& memb_nm, bool on )

set inherit bit

Show Source Code

float BpUnitSpec::SigmoidDeriv ( float act )

Show Source Code

float BpUnitSpec::SigmoidFun ( float netin )

Show Source Code

bool BaseSpec::SpecInheritsFrom ( BaseSpec* spec )

returns true if this spec is the spec, or is one of its children

Show Source Code

float BpUnitSpec::TanhDeriv ( float act )

Show Source Code

float BpUnitSpec::TanhFun ( float netin )

Show Source Code

void BaseSpec::UpdateChildren ( )

update any children

Show Source Code

void BaseSpec::UpdateMember ( BaseSpec* from, int memb_no )

copy member from given parent

Show Source Code

void BaseSpec::UpdateSpec ( )

update values from parent, and update children

Show Source Code

void UnitSpec::UpdateStateSpecs ( )

Update our corresponding state-side specs -- called in UAE, and call when specifically updating parameters -- each base class needs to define this, getting the proper specs etc

Show Source Code

void BaseSpec::UpdateSubSpecs ( )

update any subspec objects (overload me)

Show Source Code

bool BaseSpec::UpdtIsUsed ( bool gui_updt = true )

update the is_used flag and used_status variable -- returns is_used flag

Show Source Code

void BaseSpec::WhereUsed ( )

display a clickable list of objects that use this spec

Show Source Code

void BaseSpec::WhereUsed_impl ( taBase_PtrList& spec_user_list, String_Array& spec_name_list, bool child )

fill the list of objects where this spec is used - if searching for child specs of the original spec pass true (additional feedback to user)

Show Source Code

taSigLink** taOBase::addr_sig_link ( )

Show Source Code

Static Member and Method Documentation

Static Members

Static Methods


Static Method Documentation


Expert Member and Method Documentation

Expert Members

Expert Member Category: _NoCategory

Expert Member Category: taBase

Expert Methods

Expert Method Category: UserData


Expert Member Documentation

Expert Member Category: _NoCategory

UnitSpec::bias_spec_idx : int

index of bias con spec in list of specs

BaseSpec::children : BaseSpec_Group

sub-specs descending from this one and inheriting values

BpUnitSpec::err_fun : void(*)

replaced by enum -- this points to the error fun, set appropriately

BaseSpec::is_new : bool

is this spec newly created? if so, prompt user to apply it to some objects..

BaseSpec::is_used : bool

is THIS spec used - ignore usage of children

BaseSpec::min_obj_type : TypeDef*

mimimal object type required for spec -- for object that the spec actually operates on

BaseSpec::min_user_type : TypeDef*

mimimal object type required for spec -- for object that manages the spec and the user applies it to

BaseSpec::saved : NameVar_Array

values of members that are not overriding parent values - reset to these values if override is click

UnitSpec::spec_idx : int

index of this spec in list of specs

BaseSpec::unique : String_Array

string list of unique members

BaseSpec::used_status : BaseSpec::UsedStatus

for gui use only to indicate to user the usage of this spec and child specs

Expert Member Category: taBase

taOBase::owner : taBase*

pointer to owner

taOBase::user_data_ : UserDataItem_List*

storage for user data (created if needed) DO NOT ACCESS this list directly -- use the GetUserData / SetUserData etc interface!


Expert Method Documentation

Expert Method Category: UserData

UserDataItem_List* taOBase::GetUserDataList ( bool force = false )

gets the userdatalist for this class

Show Source Code


Copyright © 2017Regents of the University of Colorado, Carnegie Mellon University, Princeton University.
emergent 8.2.2