Detailed Description

A parameterized activation function (a.k.a. adaptive transfer function).

#include <GBlock.h>

Inheritance diagram for GClasses::GBlockSoftExp:

Public Member Functions
	GBlockSoftExp (double beta=0.0, size_t size=0)

	GBlockSoftExp (GDomNode *pNode)

GVec &	alpha ()
	Get the alpha vector. More...

const GVec &	alpha () const
	Get the alpha vector. More...

virtual void	backProp (GContext &ctx, const GVec &input, const GVec &output, const GVec &outBlame, GVec &inBlame) const override
	Evaluates outBlame, and adds to inBlame. (Note that it "adds to" the inBlame because multiple blocks may fork from a common source.) More...

virtual void	copyWeights (const GBlock *pSource) override
	Copy the weights from pSource to this block. (Assumes pSource is the same type of block.) More...

virtual void	diminishWeights (double amount, bool regularizeBiases) override
	Moves all weights in the direction of zero by the specified amount. More...

virtual bool	elementWise () const
	Returns true iff this block operates only on individual elements. More...

virtual void	forwardProp (GContext &ctx, const GVec &input, GVec &output) const override
	Evaluate the input, set the output. More...

virtual size_t	inputs () const override
	Returns the number of inputs this block consumes. More...

virtual void	maxNorm (double min, double max) override
	Scales weights if necessary such that the manitude of the weights (not including the bias) feeding into each unit are >= min and <= max. More...

virtual std::string	name () const override
	Returns the name of this block. More...

virtual size_t	outputs () const override
	Returns the number of outputs this block produces. More...

virtual void	perturbWeights (GRand &rand, double deviation) override
	Perturbs the weights that feed into the specifed units with Gaussian noise. The default values apply the perturbation to all units. More...

virtual void	resetWeights (GRand &rand) override
	Initialize the weights with small random values. More...

virtual void	resize (size_t inputs, size_t outputs) override
	Resizes this block. More...

virtual void	scaleWeights (double factor, bool scaleBiases) override
	Multiplies all the weights by the specified factor. More...

virtual GDomNode *	serialize (GDom *pDoc) const override
	Marshall this block into a DOM. More...

virtual void	step (double learningRate, const GVec &gradient) override
	Add the weight and bias gradient to the weights. More...

virtual BlockType	type () const override
	Returns the type of this block. More...

virtual void	updateGradient (GContext &ctx, const GVec &input, const GVec &outBlame, GVec &gradient) const override
	Updates the gradient for updating the weights by gradient descent. (Assumes the error has already been computed and deactivated.) More...

virtual size_t	vectorToWeights (const double *pVector) override
	Deserialize from a vector to the weights in this block. Return the number of elements consumed. More...

virtual size_t	weightCount () const override
	Returns the number of double-precision elements necessary to serialize the weights of this block into a vector. More...

virtual size_t	weightsToVector (double *pOutVector) const override
	Serialize the weights in this block into a vector. Return the number of elements written. More...

Public Member Functions inherited from GClasses::GBlock
	GBlock ()

	GBlock (GDomNode *pNode)

virtual	~GBlock ()

size_t	inPos () const
	Returns the offset in the previous layer's output where values are fed as input to this block. More...

virtual bool	isRecurrent () const
	Returns true iff this block is recurrent. More...

void	setInPos (size_t n)
	Sets the starting offset in the previous layer's output where values will be fed as input to this block. More...

virtual std::string	to_str () const
	Returns a string representation of this block. More...

virtual bool	usesGPU ()
	Returns true iff this block does its computations in parallel on a GPU. More...

Protected Attributes
GVec	m_alpha

double	m_beta

Protected Attributes inherited from GClasses::GBlock
size_t	m_inPos

Additional Inherited Members
Public Types inherited from GClasses::GBlock
Static Public Member Functions inherited from GClasses::GBlock
static GBlock *	deserialize (GDomNode *pNode)
	Unmarshalls the specified DOM node into a block object. More...

Protected Member Functions inherited from GClasses::GBlock
GDomNode *	baseDomNode (GDom *pDoc) const

void	basicTest ()
	Exercises some basic functionality that all blocks have in common. More...

Constructor & Destructor Documentation

GClasses::GBlockSoftExp::GBlockSoftExp	(	double	beta = `0.0`,
		size_t	size = `0`
	)

GClasses::GBlockSoftExp::GBlockSoftExp ( GDomNode * pNode )

Member Function Documentation

GVec& GClasses::GBlockSoftExp::alpha ( )

inline

Get the alpha vector.

const GVec& GClasses::GBlockSoftExp::alpha ( ) const

inline

Get the alpha vector.

virtual void GClasses::GBlockSoftExp::backProp	(	GContext &	ctx,
		const GVec &	input,
		const GVec &	output,
		const GVec &	outBlame,
		GVec &	inBlame
	)		const

overridevirtual

Evaluates outBlame, and adds to inBlame. (Note that it "adds to" the inBlame because multiple blocks may fork from a common source.)

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::copyWeights ( const GBlock * pSource )

overridevirtual

Copy the weights from pSource to this block. (Assumes pSource is the same type of block.)

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::diminishWeights	(	double	amount,
		bool	regularizeBiases
	)

overridevirtual

Moves all weights in the direction of zero by the specified amount.

Implements GClasses::GBlock.

virtual bool GClasses::GBlockSoftExp::elementWise ( ) const

inlinevirtual

Returns true iff this block operates only on individual elements.

Reimplemented from GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::forwardProp	(	GContext &	ctx,
		const GVec &	input,
		GVec &	output
	)		const

overridevirtual

Evaluate the input, set the output.

Implements GClasses::GBlock.

virtual size_t GClasses::GBlockSoftExp::inputs ( ) const

inlineoverridevirtual

Returns the number of inputs this block consumes.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::maxNorm	(	double	min,
		double	max
	)

overridevirtual

Scales weights if necessary such that the manitude of the weights (not including the bias) feeding into each unit are >= min and <= max.

Implements GClasses::GBlock.

virtual std::string GClasses::GBlockSoftExp::name ( ) const

inlineoverridevirtual

Returns the name of this block.

Implements GClasses::GBlock.

virtual size_t GClasses::GBlockSoftExp::outputs ( ) const

inlineoverridevirtual

Returns the number of outputs this block produces.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::perturbWeights	(	GRand &	rand,
		double	deviation
	)

overridevirtual

Perturbs the weights that feed into the specifed units with Gaussian noise. The default values apply the perturbation to all units.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::resetWeights ( GRand & rand )

overridevirtual

Initialize the weights with small random values.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::resize	(	size_t	inputs,
		size_t	outputs
	)

overridevirtual

Resizes this block.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::scaleWeights	(	double	factor,
		bool	scaleBiases
	)

overridevirtual

Multiplies all the weights by the specified factor.

Implements GClasses::GBlock.

virtual GDomNode* GClasses::GBlockSoftExp::serialize ( GDom * pDoc ) const

overridevirtual

Marshall this block into a DOM.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::step	(	double	learningRate,
		const GVec &	gradient
	)

overridevirtual

Add the weight and bias gradient to the weights.

Implements GClasses::GBlock.

virtual BlockType GClasses::GBlockSoftExp::type ( ) const

inlineoverridevirtual

Returns the type of this block.

Implements GClasses::GBlock.

virtual void GClasses::GBlockSoftExp::updateGradient	(	GContext &	ctx,
		const GVec &	input,
		const GVec &	outBlame,
		GVec &	gradient
	)		const

overridevirtual

Updates the gradient for updating the weights by gradient descent. (Assumes the error has already been computed and deactivated.)

Implements GClasses::GBlock.

virtual size_t GClasses::GBlockSoftExp::vectorToWeights ( const double * pVector )

overridevirtual

Deserialize from a vector to the weights in this block. Return the number of elements consumed.

Implements GClasses::GBlock.

virtual size_t GClasses::GBlockSoftExp::weightCount ( ) const

overridevirtual

Returns the number of double-precision elements necessary to serialize the weights of this block into a vector.

Implements GClasses::GBlock.

virtual size_t GClasses::GBlockSoftExp::weightsToVector ( double * pOutVector ) const

overridevirtual

Serialize the weights in this block into a vector. Return the number of elements written.

Implements GClasses::GBlock.

Member Data Documentation

GVec GClasses::GBlockSoftExp::m_alpha

protected

double GClasses::GBlockSoftExp::m_beta

protected

Detailed Description

Public Member Functions

Protected Attributes

Additional Inherited Members

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation