Detailed Description

GNeuralNet contains GLayers stacked upon each other. GLayer contains GBlocks concatenated beside each other. (GNeuralNet is a type of GBlock, so you can nest.) Each GBlock is an array of differentiable network units (artificial neurons). The user must add at least one GBlock to each GLayer.

#include <GNeuralNet.h>

Inheritance diagram for GClasses::GNeuralNet:

Public Member Functions
	GNeuralNet ()

	GNeuralNet (GDomNode *pNode)

virtual	~GNeuralNet ()

void	add (GBlock *pBlock)
	Adds a block as a new layer to this neural network. More...

void	add (GBlock a, GBlock b)

void	add (GBlock a, GBlock b, GBlock *c)

void	add (GBlock a, GBlock b, GBlock c, GBlock d)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock *e)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f, GBlock *g)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f, GBlock g, GBlock h)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f, GBlock g, GBlock h, GBlock *i)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f, GBlock g, GBlock h, GBlock i, GBlock j)

void	add (GBlock a, GBlock b, GBlock c, GBlock d, GBlock e, GBlock f, GBlock g, GBlock h, GBlock i, GBlock j, GBlock *k)

void	align (const GNeuralNet &that)
	Swaps nodes in hidden layers of this neural network to align with those in that neural network, as determined using bipartite matching. (This might be done, for example, before averaging weights together.) More...

virtual void	backProp (GContext &ctx, const GVec &input, const GVec &output, const GVec &outBlame, GVec &inBlame) const override
	Evaluates outBlame, computes inBlame. For efficiency reasons, as a special case, if inBlame.data() == outBlame.data(), then inBlame will not be computed. More...

void	concat (GBlock *pBlock, size_t inPos=0)
	Concatenates a block to the last (output-most) layer in this neural network. (inPos specifies the starting position of the inputs into this block.) More...

void	copyPrediction (GVec &out)
	This method assumes forwardProp has been called. It copies the predicted vector into pOut. More...

void	copyStructure (const GNeuralNet *pOther)
	Makes this object into a deep copy of pOther, including layers, nodes, settings and weights. More...

virtual void	copyWeights (const GBlock *pOther) override
	Copy the weights from pOther. It is assumed (but not checked) that pOther already is a GNeuralNet with the same structure as this one. This method is faster than copyStructure. More...

virtual void	diminishWeights (double amount, bool regularizeBiases=true) override
	Diminishes all weights in the network by the specified amount. This can be used to implemnet L1 regularization, which promotes sparse representations. That is, it makes many of the weights approach zero. More...

virtual void	forwardProp (GContext &ctx, const GVec &input, GVec &output) const override
	Performs principal component analysis (without reducing dimensionality) on the features to shift the variance of the data to the first few columns. Adjusts the weights on the input layer accordingly, such that the network output remains the same. Returns the transformed feature matrix. More...

void	forwardProp_training (GContext &ctx, const GVec &input, GVec &output) const
	Evaluates input, computes output. This method differs from forwardProp in that it unfolds recurrent blocks through time. More...

void	init (size_t inputs, size_t outputs, GRand &rand)
	Calls resize, then resetWeights. More...

virtual size_t	inputs () const override
	Returns the number of inputs this layer consumes. More...

void	invertNode (size_t layer, size_t node)
	Inverts the weights of the specified node, and adjusts the weights in the next layer (if there is one) such that this will have no effect on the output of the network. (Assumes this model is already trained.) More...

GLayer &	layer (size_t i)
	Returns the specified layer. More...

const GLayer &	layer (size_t i) const

size_t	layerCount () const
	Returns the number of layers in this neural net. (Layers within neural networks embedded within this one are not counted.) More...

virtual void	maxNorm (double min, double max) override
	Scales weights if necessary such that the magnitude of the weights (not including the bias) feeding into each unit are >= min and <= max. More...

double	measureLoss (const GMatrix &features, const GMatrix &labels, double *pOutSAE=nullptr)
	Measures the loss with respect to some data. Returns sum-squared error. if pOutSAE is not nullptr, then sum-absolute error will be storead where it points. As a special case, if labels have exactly one categorical column, then it will be assumed that the maximum output unit of this neural network represents a categorical prediction, and sum hamming loss will be returned. More...

virtual std::string	name () const override
	Returns the name of this block. More...

GContextNeuralNet *	newContext (GRand &rand) const
	Allocates a new GContextNeuralNet object, which can be used to train or predict with this neural net. (Behavior is undefined if you add or modify any layers after you call newContext.) More...

GLayer &	outputLayer ()
	Returns a reference to the last layer. More...

const GLayer &	outputLayer () const

virtual size_t	outputs () const override
	Returns the number of outputs this layer produces. More...

virtual void	perturbWeights (GRand &rand, double deviation) override
	Perturbs all weights in the network by a random normal offset with the specified deviation. More...

void	printWeights (std::ostream &stream)
	Prints weights in a human-readable format. More...

void	recount ()
	Recounts the number of weights. More...

virtual void	resetWeights (GRand &rand) override
	Initialize the weights, usually with small random values. More...

virtual void	resize (size_t inputs, size_t outputs) override
	Resizes this layer. More...

virtual void	scaleWeights (double factor, bool scaleBiases=true) override
	Multiplies all weights in the network by the specified factor. This can be used to implement L2 regularization, which prevents weight saturation. The factor for L2 regularization should be less than 1.0, but most likely somewhat close to 1. More...

GDomNode *	serialize (GDom *pDoc) const override
	Marshal this object into a dom node. More...

virtual void	step (double learningRate, const GVec &gradient) override
	Take a step to descend the gradient by updating the weights. More...

void	swapNodes (size_t layer, size_t a, size_t b)
	Swaps two nodes in the specified layer. If layer specifies one of the hidden layers, then this will have no net effect on the output of the network. (Assumes this model is already trained.) More...

virtual std::string	to_str () const override
	Returns a string representation of this object. More...

std::string	to_str (const std::string &line_prefix) const
	Same as to_str, but it lets the use specify a string to prepend to each line. More...

virtual BlockType	type () const override
	Returns the type of this layer. More...

virtual void	updateGradient (GContext &ctx, const GVec &x, const GVec &outBlame, GVec &inBlame) const override
	Updates the gradient. More...

virtual size_t	vectorToWeights (const double *pWeights) override
	Sets all the weights from an array of doubles. The number of doubles in the array can be determined by calling weightCount(). More...

virtual size_t	weightCount () const override
	Returns the number of weights. More...

virtual size_t	weightsToVector (double *pOutWeights) const override
	Serializes the network weights into an array of doubles. The number of doubles in the array can be determined by calling weightCount(). More...

Public Member Functions inherited from GClasses::GBlock
	GBlock ()

	GBlock (GDomNode *pNode)

virtual	~GBlock ()

virtual bool	elementWise () const
	Returns true iff this block operates only on individual elements. More...

size_t	inPos () const
	Returns the offset in the previous layer's output where values are fed as input to this block. More...

virtual bool	isRecurrent () const
	Returns true iff this block is recurrent. More...

void	setInPos (size_t n)
	Sets the starting offset in the previous layer's output where values will be fed as input to this block. More...

virtual bool	usesGPU ()
	Returns true iff this block does its computations in parallel on a GPU. More...

Protected Attributes
std::vector< GLayer * >	m_layers

size_t	m_weightCount

Protected Attributes inherited from GClasses::GBlock
size_t	m_inPos

Additional Inherited Members
Public Types inherited from GClasses::GBlock
Static Public Member Functions inherited from GClasses::GBlock
static GBlock *	deserialize (GDomNode *pNode)
	Unmarshalls the specified DOM node into a block object. More...

Protected Member Functions inherited from GClasses::GBlock
GDomNode *	baseDomNode (GDom *pDoc) const

void	basicTest ()
	Exercises some basic functionality that all blocks have in common. More...

Constructor & Destructor Documentation

GClasses::GNeuralNet::GNeuralNet ( )

GClasses::GNeuralNet::GNeuralNet ( GDomNode * pNode )

virtual GClasses::GNeuralNet::~GNeuralNet ( )

virtual

Member Function Documentation

void GClasses::GNeuralNet::add ( GBlock * pBlock )

Adds a block as a new layer to this neural network.

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f,
		GBlock *	g
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f,
		GBlock *	g,
		GBlock *	h
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f,
		GBlock *	g,
		GBlock *	h,
		GBlock *	i
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f,
		GBlock *	g,
		GBlock *	h,
		GBlock *	i,
		GBlock *	j
	)

inline

void GClasses::GNeuralNet::add	(	GBlock *	a,
		GBlock *	b,
		GBlock *	c,
		GBlock *	d,
		GBlock *	e,
		GBlock *	f,
		GBlock *	g,
		GBlock *	h,
		GBlock *	i,
		GBlock *	j,
		GBlock *	k
	)

inline

void GClasses::GNeuralNet::align ( const GNeuralNet & that )

Swaps nodes in hidden layers of this neural network to align with those in that neural network, as determined using bipartite matching. (This might be done, for example, before averaging weights together.)

virtual void GClasses::GNeuralNet::backProp	(	GContext &	ctx,
		const GVec &	input,
		const GVec &	output,
		const GVec &	outBlame,
		GVec &	inBlame
	)		const

overridevirtual

Evaluates outBlame, computes inBlame. For efficiency reasons, as a special case, if inBlame.data() == outBlame.data(), then inBlame will not be computed.

void GClasses::GNeuralNet::concat	(	GBlock *	pBlock,
		size_t	inPos = `0`
	)

void GClasses::GNeuralNet::forwardProp_training	(	GContext &	ctx,
		const GVec &	input,
		GVec &	output
	)		const

void GClasses::GNeuralNet::init	(	size_t	inputs,
		size_t	outputs,
		GRand &	rand
	)

void GClasses::GNeuralNet::invertNode	(	size_t	layer,
		size_t	node
	)

double GClasses::GNeuralNet::measureLoss	(	const GMatrix &	features,
		const GMatrix &	labels,
		double *	pOutSAE = `nullptr`
	)

void GClasses::GNeuralNet::swapNodes	(	size_t	layer,
		size_t	a,
		size_t	b
	)

Detailed Description

Public Member Functions

Protected Attributes

Additional Inherited Members

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation