GClasses
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
GClasses::GBlock Class Referenceabstract

Detailed Description

Represents a block of network units (artificial neurons) in a neural network.

#include <GBlock.h>

Inheritance diagram for GClasses::GBlock:
GClasses::GBlockConvolutional1D GClasses::GBlockConvolutional2D GClasses::GBlockFeatureSelector GClasses::GBlockFuzzy GClasses::GBlockLinear GClasses::GBlockRecurrent GClasses::GBlockRestrictedBoltzmannMachine GClasses::GBlockSoftExp GClasses::GBlockSparse GClasses::GBlockWeightless GClasses::GNeuralNet

Public Types

Public Member Functions

 GBlock ()
 
 GBlock (GDomNode *pNode)
 
virtual ~GBlock ()
 
virtual void backProp (GContext &ctx, const GVec &input, const GVec &output, const GVec &outBlame, GVec &inBlame) const =0
 Evaluates outBlame, and adds to inBlame. (Note that it "adds to" the inBlame because multiple blocks may fork from a common source.) More...
 
virtual void copyWeights (const GBlock *pSource)=0
 Copy the weights from pSource to this block. (Assumes pSource is the same type of block.) More...
 
virtual void diminishWeights (double amount, bool regularizeBiases)=0
 Moves all weights in the direction of zero by the specified amount. More...
 
virtual bool elementWise () const
 Returns true iff this block operates only on individual elements. More...
 
virtual void forwardProp (GContext &ctx, const GVec &input, GVec &output) const =0
 Evaluates the input, sets the output. More...
 
size_t inPos () const
 Returns the offset in the previous layer's output where values are fed as input to this block. More...
 
virtual size_t inputs () const =0
 Returns the number of inputs this block consumes. More...
 
virtual bool isRecurrent () const
 Returns true iff this block is recurrent. More...
 
virtual void maxNorm (double min, double max)=0
 Scales weights if necessary such that the manitude of the weights (not including the bias) feeding into each unit are >= min and <= max. More...
 
virtual std::string name () const =0
 Returns the name of this block in the form of a string. More...
 
virtual size_t outputs () const =0
 Returns the number of outputs this block produces. More...
 
virtual void perturbWeights (GRand &rand, double deviation)=0
 Perturbs the weights that feed into the specifed units with Gaussian noise. The default values apply the perturbation to all units. More...
 
virtual void resetWeights (GRand &rand)=0
 Initialize the weights, usually with small random values. More...
 
virtual void resize (size_t inputs, size_t outputs)=0
 Resizes this block. More...
 
virtual void scaleWeights (double factor, bool scaleBiases)=0
 Multiplies all the weights by the specified factor. More...
 
virtual GDomNode * serialize (GDom *pDoc) const =0
 Marshall this block into a DOM. More...
 
void setInPos (size_t n)
 Sets the starting offset in the previous layer's output where values will be fed as input to this block. More...
 
virtual void step (double learningRate, const GVec &gradient)=0
 Add the weight and bias gradient to the weights. More...
 
virtual std::string to_str () const
 Returns a string representation of this block. More...
 
virtual BlockType type () const =0
 Returns the type of this block. More...
 
virtual void updateGradient (GContext &ctx, const GVec &input, const GVec &outBlame, GVec &gradient) const =0
 Evaluate the input and outBlame, update the gradient for updating the weights by gradient descent. More...
 
virtual bool usesGPU ()
 Returns true iff this block does its computations in parallel on a GPU. More...
 
virtual size_t vectorToWeights (const double *pVector)=0
 Deserialize from a vector to the weights in this block. Return the number of elements consumed. More...
 
virtual size_t weightCount () const =0
 Returns the number of double-precision elements necessary to serialize the weights of this block into a vector. More...
 
virtual size_t weightsToVector (double *pOutVector) const =0
 Serialize the weights in this block into a vector. Return the number of elements written. More...
 

Static Public Member Functions

static GBlock * deserialize (GDomNode *pNode)
 Unmarshalls the specified DOM node into a block object. More...
 

Protected Member Functions

GDomNode * baseDomNode (GDom *pDoc) const
 
void basicTest ()
 Exercises some basic functionality that all blocks have in common. More...
 

Protected Attributes

size_t m_inPos
 

Member Enumeration Documentation

enum GClasses::GBlock::BlockType
Enumerator
block_neuralnet 
block_identity 
block_tanh 
block_scaledtanh 
block_logistic 
block_bentidentity 
block_sigexp 
block_gaussian 
block_sine 
block_rectifier 
block_leakyrectifier 
block_softplus 
block_softroot 
block_linear 
block_sparse 
block_featureselector 
block_fuzzy 
block_restrictedboltzmannmachine 
block_convolutional1d 
block_convolutional2d 
block_softexp 
block_scalarsum 
block_scalarproduct 
block_switch 
block_maxpooling 
block_allpairings 
block_lstm 
block_gru 

Constructor & Destructor Documentation

GClasses::GBlock::GBlock ( )
GClasses::GBlock::GBlock ( GDomNode *  pNode)
virtual GClasses::GBlock::~GBlock ( )
inlinevirtual

Member Function Documentation

virtual void GClasses::GBlock::backProp ( GContext &  ctx,
const GVec &  input,
const GVec &  output,
const GVec &  outBlame,
GVec &  inBlame 
) const
pure virtual

Evaluates outBlame, and adds to inBlame. (Note that it "adds to" the inBlame because multiple blocks may fork from a common source.)

Implemented in GClasses::GBlockRecurrent, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GNeuralNet, GClasses::GBlockScalarProduct, and GClasses::GBlockScalarSum.

GDomNode* GClasses::GBlock::baseDomNode ( GDom *  pDoc) const
protected
void GClasses::GBlock::basicTest ( )
protected

Exercises some basic functionality that all blocks have in common.

virtual void GClasses::GBlock::copyWeights ( const GBlock *  pSource)
pure virtual

Copy the weights from pSource to this block. (Assumes pSource is the same type of block.)

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

static GBlock* GClasses::GBlock::deserialize ( GDomNode *  pNode)
static

Unmarshalls the specified DOM node into a block object.

virtual void GClasses::GBlock::diminishWeights ( double  amount,
bool  regularizeBiases 
)
pure virtual

Moves all weights in the direction of zero by the specified amount.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual bool GClasses::GBlock::elementWise ( ) const
inlinevirtual

Returns true iff this block operates only on individual elements.

Reimplemented in GClasses::GBlockSoftExp, and GClasses::GBlockActivation.

virtual void GClasses::GBlock::forwardProp ( GContext &  ctx,
const GVec &  input,
GVec &  output 
) const
pure virtual

Evaluates the input, sets the output.

Implemented in GClasses::GBlockRecurrent, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GNeuralNet, GClasses::GBlockScalarProduct, and GClasses::GBlockScalarSum.

size_t GClasses::GBlock::inPos ( ) const
inline

Returns the offset in the previous layer's output where values are fed as input to this block.

virtual size_t GClasses::GBlock::inputs ( ) const
pure virtual

Returns the number of inputs this block consumes.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

virtual bool GClasses::GBlock::isRecurrent ( ) const
inlinevirtual

Returns true iff this block is recurrent.

Reimplemented in GClasses::GBlockRecurrent.

virtual void GClasses::GBlock::maxNorm ( double  min,
double  max 
)
pure virtual

Scales weights if necessary such that the manitude of the weights (not including the bias) feeding into each unit are >= min and <= max.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual std::string GClasses::GBlock::name ( ) const
pure virtual

Returns the name of this block in the form of a string.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockSoftRoot, GClasses::GBlockSoftPlus, GClasses::GBlockLeakyRectifier, GClasses::GBlockRectifier, GClasses::GBlockSine, GClasses::GBlockGaussian, GClasses::GBlockSigExp, GClasses::GBlockBentIdentity, GClasses::GBlockLogistic, GClasses::GBlockScaledTanh, GClasses::GBlockTanh, GClasses::GBlockIdentity, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

virtual size_t GClasses::GBlock::outputs ( ) const
pure virtual

Returns the number of outputs this block produces.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

virtual void GClasses::GBlock::perturbWeights ( GRand &  rand,
double  deviation 
)
pure virtual

Perturbs the weights that feed into the specifed units with Gaussian noise. The default values apply the perturbation to all units.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual void GClasses::GBlock::resetWeights ( GRand &  rand)
pure virtual

Initialize the weights, usually with small random values.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual void GClasses::GBlock::resize ( size_t  inputs,
size_t  outputs 
)
pure virtual

Resizes this block.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

virtual void GClasses::GBlock::scaleWeights ( double  factor,
bool  scaleBiases 
)
pure virtual

Multiplies all the weights by the specified factor.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual GDomNode* GClasses::GBlock::serialize ( GDom *  pDoc) const
pure virtual

Marshall this block into a DOM.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockActivation, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

void GClasses::GBlock::setInPos ( size_t  n)
inline

Sets the starting offset in the previous layer's output where values will be fed as input to this block.

virtual void GClasses::GBlock::step ( double  learningRate,
const GVec &  gradient 
)
pure virtual

Add the weight and bias gradient to the weights.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual std::string GClasses::GBlock::to_str ( ) const
virtual

Returns a string representation of this block.

Reimplemented in GClasses::GNeuralNet.

virtual BlockType GClasses::GBlock::type ( ) const
pure virtual

Returns the type of this block.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockAllPairings, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GBlockSoftRoot, GClasses::GBlockSoftPlus, GClasses::GBlockLeakyRectifier, GClasses::GBlockRectifier, GClasses::GBlockSine, GClasses::GBlockGaussian, GClasses::GBlockSigExp, GClasses::GBlockBentIdentity, GClasses::GBlockLogistic, GClasses::GBlockScaledTanh, GClasses::GBlockTanh, GClasses::GBlockIdentity, GClasses::GMaxPooling2D, GClasses::GBlockSwitch, GClasses::GBlockScalarProduct, GClasses::GBlockScalarSum, and GClasses::GNeuralNet.

virtual void GClasses::GBlock::updateGradient ( GContext &  ctx,
const GVec &  input,
const GVec &  outBlame,
GVec &  gradient 
) const
pure virtual

Evaluate the input and outBlame, update the gradient for updating the weights by gradient descent.

Implemented in GClasses::GBlockRecurrent, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual bool GClasses::GBlock::usesGPU ( )
inlinevirtual

Returns true iff this block does its computations in parallel on a GPU.

virtual size_t GClasses::GBlock::vectorToWeights ( const double *  pVector)
pure virtual

Deserialize from a vector to the weights in this block. Return the number of elements consumed.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual size_t GClasses::GBlock::weightCount ( ) const
pure virtual

Returns the number of double-precision elements necessary to serialize the weights of this block into a vector.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

virtual size_t GClasses::GBlock::weightsToVector ( double *  pOutVector) const
pure virtual

Serialize the weights in this block into a vector. Return the number of elements written.

Implemented in GClasses::GBlockGRU, GClasses::GBlockLSTM, GClasses::GBlockConvolutional2D, GClasses::GBlockConvolutional1D, GClasses::GBlockRestrictedBoltzmannMachine, GClasses::GBlockFuzzy, GClasses::GBlockFeatureSelector, GClasses::GBlockSparse, GClasses::GBlockLinear, GClasses::GBlockSoftExp, GClasses::GNeuralNet, and GClasses::GBlockWeightless.

Member Data Documentation

size_t GClasses::GBlock::m_inPos
protected