MSERNNet Class Reference

A recurrent neural network regression model that learns with Back Propagation Through Time and assumes the MSE error functional. More...

#include <MSERNNet.h>

Inheritance diagram for MSERNNet:

List of all members.

Public Member Functions

MSERNNet ()

Creates an empty MSE recurrent neural network.

MSERNNet (Array< int >)

Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con".

MSERNNet (Array< int >, Array< double >)

Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con" and a set of weights given by weight matrix "wei".

MSERNNet (std::string)

Creates a recurrent neural network by reading the necessary information from a file named "filename".

void includeWarmUp (unsigned WUP=0)

Sets the length of the warmup sequence.

void model (const Array< double > &input)

Feed a data series to the model.

void model (const Array< double > &input, Array< double > &output)

Feed a data series to the model. The output (i.e., the time series of activations of the output neurons) is copies into the output# buffer.

double errorPercentage (const Array< double > &input, const Array< double > &target)

Evaluates the error percentage of the network output compared to given target data.

double error (Model &model, const Array< double > &input, const Array< double > &target)

Evaluates the mean squared error of the network output compared to given target data.

double errorDerivative (Model &model, const Array< double > &input, const Array< double > &target, Array< double > &derivative)

Evaluates the derivative of the mean squared error.

double meanSquaredError (const Array< double > &input, const Array< double > &target)

same as error

Array< int > & getConnections ()

Returns the connection Matrix.

Array< double > & getWeights ()

Returns the weight matrix.

void initWeights (double, double)

Initializes the weights of the net.

void initWeights (long, double, double)

Initializes the random number generator and the weights of the net.

void setStructure (Array< int > &)

Based on a given connection matrix a network is created.

void setStructure (Array< int > &, Array< double > &)

Based on given connection and weight matrices a network is created.

void setStructure (Array< double > &)

Replaces the existing weight matrix by a new matrix.

void setStructure (std::string)

The current structure of the network is replaced by a new one that is based on the information in file "filename".

void write (std::string)

Writes the structure of the network to a file named "filename".

void setStructur (unsigned in, unsigned hidden, unsigned out, unsigned memory=1, bool layered=false, bool recurrentInputs=true, bool bias=true, bool elman=false, bool previousInputs=false)

Creates a connection matrix for a recurrent network.

void setHistory (const Array< double > &)

Sets the history of the neuron states to certain values.

Array< double > getHistory ()

Returns the history of the neuron states.

Protected Member Functions

virtual double g (double)

Activation function of all neurons.

virtual double dg (double)

Computes the derivative of g(a) for all neurons.

void init0 ()

Initializes some internal variables.

void writeParameters ()

Writes the values of the weights stored in the weight matrix to the parameter vector w of ModelInterface.

void readParameters ()

Reads the values of the parameter vector w of ModelInterface and stores these values in the weight matrix.

void writeGradient (Array< double > &derivative)

void prepareTime (unsigned t)

Performs some initializations that are necessary to process the time series.

void processTimeSeries (const Array< double > &input)

Processes a whole time series.

void processTimeStep ()

Processes one input pattern.

void calcGradBPTT ()

Protected Attributes

unsigned numberOfNeurons

The total number of neurons of the network (input, output and hidden).

unsigned numberOfParameters

The absolute number of parameters of the network (equal to the number of weights).

unsigned time

the variable that counts the time (backwards!) when processing a data series

unsigned delay

unsigned episode

unsigned warmUpLength

ArrayTable< int > connectionMatrix

ArrayTable< double > weightMatrix

ArrayTable< int > delMask

ArrayTable< double > stimulus

ArrayTable< double > Y

ArrayTable< double > err

This array stores the errors of the neurons for every input pattern.

ArrayTable< double > delta

Stores the local delta values which are used to evaluate the gradient.

ArrayTable< double > dEw

Detailed Description

A recurrent neural network regression model that learns with Back Propagation Through Time and assumes the MSE error functional.

This class defines a recurrent neural network regression model. The input and output arrays should, as usual, be 2-dimensional and have the dimensionality (batch-length,number-of-neurons). The only difference is that here, the batch corresponds to a time series instead of independent data points. The gradient is calculated via BackPropTroughTime (BPTT). This model implies the MSE as the error functional.

The class can handle arbitrary network architectures. Feed-forward connections (with time delay `zero') as well as connections of any time delay can be realized --- see setStructure() for details of how to define the structure.

Things to note!:

(1) All neurons are sigmoidal! So please transform inputs and outputs to take this into account. The reason is that internally, no differences between input, hidden, or output neurons is made.

(2) The initialization of the state history is an important issue for dynamic systems like an RNN. Please read the documentation of setWarmUpLentgh().

(3) Online learning can, in principle, be realized by having a batch-length of zero (the input/output arrays still have to be 2-dimensional, i.e., of dimension (1,number-of-neurons)) and by setting the WarmUpLength to zero. Note though that this is quite inefficient compared to batch-learning (because the BPTT does not save you any calculation time) and that cross-talk will make learning _very_ difficult. Maybe, even when online learning, you should still use a reasonable batch size (say 100), feed these batches sequentially, and set the WarmUpLength to zero.

(Internals: the only core functions are `processTimeSeries' and `calcGradBPTT' in the cpp-file. Almost all the rest it `utility' stuff.)

Author:: M. Toussaint

Date:: 2000

Changes:: none

Status:: stable

Examples:: simpleMSERNNet.cpp, and simpleRNNet.cpp.

Definition at line 113 of file MSERNNet.h.

Constructor & Destructor Documentation

MSERNNet::MSERNNet ( )

Creates an empty MSE recurrent neural network.

Examples:: simpleMSERNNet.cpp, and simpleRNNet.cpp.

Definition at line 42 of file MSERNNet.cpp.

References init0().

MSERNNet::MSERNNet ( Array< int > con )

Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con".

Definition at line 47 of file MSERNNet.cpp.

References init0(), and setStructure().

MSERNNet::MSERNNet	(	Array< int >	con,
		Array< double >	wei
	)

Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con" and a set of weights given by weight matrix "wei".

Definition at line 53 of file MSERNNet.cpp.

References init0(), and setStructure().

MSERNNet::MSERNNet ( std::string filename )

Creates a recurrent neural network by reading the necessary information from a file named "filename".

Definition at line 59 of file MSERNNet.cpp.

References init0(), and setStructure().

Member Function Documentation

void MSERNNet::calcGradBPTT ( ) [protected]

Performs backpropagation through time to calculate the derivative of the error with respect to the weights. The results are stored to dEw.

Definition at line 532 of file MSERNNet.cpp.

References connectionMatrix, delay, delMask, delta, dEw, dg(), episode, err, i, numberOfNeurons, weightMatrix, and Y.

Referenced by errorDerivative().

double MSERNNet::dg ( double ga ) [protected, virtual]

Computes the derivative of g(a) for all neurons.

Examples:: simpleRNNet.cpp.

Definition at line 402 of file MSERNNet.cpp.

Referenced by calcGradBPTT().

double MSERNNet::error	(	Model &	model,
		const Array< double > &	input,
		const Array< double > &	target
	)			`[virtual]`

Evaluates the mean squared error of the network output compared to given target data.

Given the patterns in input and the patterns in target for the comparison to the outputs of the network, the mean squared error $E$ as given in the details is calculated.

Keep in mind, that if you have defined a warmup length greater than zero for the network before, the first warmup-length input patterns are not used for the calculation of the error. Of course, the number of patterns in input must be greater than the warmup-length otherwise the method will with failure.

Parameters:

	input	Input patterns for the network.
	target	Target patterns used for comparison to the outputs of the network.

Returns:: The mean squared error .

Implements ErrorFunction.

Definition at line 94 of file MSERNNet.cpp.

References err, i, model(), numberOfNeurons, warmUpLength, and Y.

Referenced by errorDerivative(), and meanSquaredError().

double MSERNNet::errorDerivative	(	Model &	model,
		const Array< double > &	input,
		const Array< double > &	target,
		Array< double > &	derivative
	)			`[virtual]`

Evaluates the derivative of the mean squared error.

Given the patterns in input and the patterns in target for the comparison to the outputs of the network, the derivatives of the mean squared error $E$ as given in the details are calculated and the results are stored in the parameter derivative. Additionally, the mean squared error itself is returned. Keep in mind, that if you have defined a warmup length greater than zero for the network before, the first $warmup-length$ input patterns are not used for the calculation of the error.

Parameters:

	input	Input patterns for the network.
	target	Target patterns used for comparison to the outputs of the network.

Reimplemented from ErrorFunction.

Definition at line 179 of file MSERNNet.cpp.

References calcGradBPTT(), error(), warmUpLength, and writeGradient().

double MSERNNet::errorPercentage	(	const Array< double > &	input,
		const Array< double > &	target
	)

Evaluates the error percentage of the network output compared to given target data.

Given the patterns in input and the patterns in target for the comparison to the outputs of the network, the error percentage is calculated, i.e. the mean squared error $E$ as given in the details is calculated and the error percentage is then given as

$EP = \frac{100}{(outmax - outmin)^2} \ast E$

where $outmax$ is the maximum value that can be stored in a double value and $outmin$ is the corresponding minimum value. Keep in mind, that if you have defined a warmup length greater than zero for the network before, the first $warmup-length$ input patterns are not used for the calculation of the error. Of course, the number of patterns in input must be greater than the warmup-length otherwise the method will exit with an exception.

Definition at line 130 of file MSERNNet.cpp.

References err, i, model(), numberOfNeurons, warmUpLength, and Y.

double MSERNNet::g ( double a ) [protected, virtual]

Activation function of all neurons.

Examples:: simpleRNNet.cpp.

Definition at line 395 of file MSERNNet.cpp.

Referenced by processTimeStep().

Array< int > & MSERNNet::getConnections ( )

Returns the connection Matrix.

The 3-dimensional connection matrix of the network is returned. The first dimension is used for the number of delays, with each delay having a 2-dimensional connection matrix as known from other networks.

Returns:: The 3-dimensional connection matrix.

Definition at line 190 of file MSERNNet.cpp.

References connectionMatrix.

Referenced by write().

Array< double > MSERNNet::getHistory ( )

Returns the history of the neuron states.

This returns the current history of neuron activations. Use it for setHistory#. See the explanations of the history format there.

Returns:: The current content of the history.

Author:: M. Toussaint

Date:: 2000

Changes: none

Status: stable

Definition at line 376 of file MSERNNet.cpp.

References delay, i, numberOfNeurons, and Y.

Array< double > & MSERNNet::getWeights ( )

Returns the weight matrix.

The 3-dimensional weight matrix of the network is returned. The first dimension is used for the number of delays, with each delay having a 2-dimensional weight matrix as known from other networks.

Returns:: The 3-dimensional weight matrix.

Definition at line 195 of file MSERNNet.cpp.

References weightMatrix.

Referenced by write().

void MSERNNet::includeWarmUp ( unsigned WUP = 0 )

Sets the length of the warmup sequence.

Usually, when processing a new data series (e.g., by calling model(...), error(...), or anything like that) then all the `states' of the network are reset to zero. By `states' I mean the buffered activations to which time-delayed synapses refer to. Effectively, this means one assumes a zero activation history.

The advantage of this is, that it makes the model behavior well defined. The disadvantage is that you can't predict a time series well with a zero history. Thus, one should use the first few inputs of a data series to initialize the network, i.e., to let it converge into a `normal' dynamic state from which prediction of new data is possible. This phase is called the warmup phase. (Internally, the warmup phase differes from subsequent data only in that it does not contributed to the error functional and does not induce an error gradient.)

With this function you can set to time span of the warmup phase (which also means the amount of data that you `waste' for the warmup instead of the learning.)

NOTE: Sometimes, e.g., when feeding a sequence of data sequences to the model, it is desirable not to reset the internal states to zero. This is the case when you set a WUP to zero.

Parameters:

WUP

Length of the warmup sequence, the default value is "0".

Definition at line 66 of file MSERNNet.cpp.

References warmUpLength.

void MSERNNet::init0 ( ) [protected]

Initializes some internal variables.

Definition at line 409 of file MSERNNet.cpp.

References delay, delta, episode, err, numberOfNeurons, numberOfParameters, time, warmUpLength, and Y.

Referenced by MSERNNet().

void MSERNNet::initWeights	(	long	seed,
		double	min,
		double	max
	)

Initializes the random number generator and the weights of the net.

same as Rng::seed(seed); initWeights(min, max);#

Parameters:

	seed	The initialization value for the random number generator.
	min	The minimum possible initialization value.
	max	The maximum possible initialization value.

Returns:: None.

Definition at line 200 of file MSERNNet.cpp.

References initWeights().

void MSERNNet::initWeights	(	double	low,
		double	up
	)

Initializes the weights of the net.

The weights of the network for all delays are initialized by uniformally distributed numbers between low and up.

Parameters:

	low	The minimum possible initialization value.
	up	The maximum possible initialization value.

Returns:: None.

Examples:: simpleMSERNNet.cpp.

Definition at line 206 of file MSERNNet.cpp.

References connectionMatrix, delay, i, numberOfNeurons, weightMatrix, and writeParameters().

Referenced by initWeights().

double MSERNNet::meanSquaredError	(	const Array< double > &	input,
		const Array< double > &	target
	)

same as error

Parameters:

	input	Input patterns for the network.
	target	Target patterns used for comparison to the outputs of the network.

Returns:: The mean squared error .

Definition at line 173 of file MSERNNet.cpp.

References error().

void MSERNNet::model	(	const Array< double > &	input,
		Array< double > &	output
	)			`[virtual]`

Feed a data series to the model. The output (i.e., the time series of activations of the output neurons) is copies into the output# buffer.

Parameters:

	input	Input patterns for the network.
	output	Used to store the outputs of the network.

Implements Model.

Definition at line 81 of file MSERNNet.cpp.

References i, model(), numberOfNeurons, and Y.

void MSERNNet::model ( const Array< double > & input )

Feed a data series to the model.

The output of the network is only stored internally. input must be 2-dimensional, where the first index refers to the time, the second to the neuron. To view the outputs, please use method model(const Array<double>& input,Array<double>& output) instead.

Parameters:

input

Input patterns for the network.

Returns:: None.

Definition at line 71 of file MSERNNet.cpp.

References processTimeSeries(), warmUpLength, and Y.

Referenced by error(), errorPercentage(), and model().

void MSERNNet::prepareTime ( unsigned t ) [protected]

Performs some initializations that are necessary to process the time series.

Definition at line 452 of file MSERNNet.cpp.

References delay, delta, episode, err, i, numberOfNeurons, time, and Y.

Referenced by processTimeSeries().

void MSERNNet::processTimeSeries ( const Array< double > & input ) [protected]

Processes a whole time series.

After processing the output can be found in the variable Y.

Definition at line 484 of file MSERNNet.cpp.

References i, numberOfNeurons, prepareTime(), processTimeStep(), readParameters(), and stimulus.

Referenced by model().

void MSERNNet::processTimeStep ( ) [protected]

Processes one input pattern.

Definition at line 504 of file MSERNNet.cpp.

References delay, delMask, g(), i, numberOfNeurons, stimulus, time, weightMatrix, and Y.

Referenced by processTimeSeries().

void MSERNNet::readParameters ( ) [protected]

Reads the values of the parameter vector w of ModelInterface and stores these values in the weight matrix.

Definition at line 431 of file MSERNNet.cpp.

References connectionMatrix, delay, i, numberOfNeurons, Model::parameter, and weightMatrix.

Referenced by processTimeSeries().

void MSERNNet::setHistory ( const Array< double > & Ystate )

Sets the history of the neuron states to certain values.

In case you chose no warmup phase (warmup length set to zero) then, the network will not be reinitialized when new data is feeded in. This might make sense if previous data was processes and the current state of the model is well-defined and can ne used for further predictions.

Alternatively, with this function, you can explicitly set the history of the network -- I guess this makes only sense when you have recorded a history earlier or if you set it explicitly to zero.

The first index of the array corresponds to the index of the neuron (and must have dimension equal to the total number of neurons). The second index corresponds to the time IN REVERSE ORDER (the index denotes `the time before now') and it must have the dimensionality delay#.

Parameters:

Ystate

The new values for the history.

Returns:: None.

Definition at line 364 of file MSERNNet.cpp.

References delay, i, numberOfNeurons, and Y.

void MSERNNet::setStructur	(	unsigned	in,
		unsigned	hidden,
		unsigned	out,
		unsigned	memory = `1`,
		bool	layered = `false`,
		bool	recurrentInputs = `true`,
		bool	bias = `true`,
		bool	elman = `false`,
		bool	previousInputs = `false`
	)

Creates a connection matrix for a recurrent network.

Changes: none

Status: stable

void MSERNNet::setStructure ( std::string filename )

The current structure of the network is replaced by a new one that is based on the information in file "filename".

A file is used to replace the current structure of the network. This file must have the following content: The first line contains the number of time steps used, followed by $number\ of\ time\ steps$ connection matrices, followed by $number\ of\ time\ steps$ weight matrices.

Parameters:

filename

Name of the file that contains the information for the modification of the structure.

Returns:: None.

Definition at line 260 of file MSERNNet.cpp.

References i, and setStructure().

void MSERNNet::setStructure ( Array< double > & wei )

Replaces the existing weight matrix by a new matrix.

The current weight matrix is replaced by the new weight matrix wei. This means NOT, that the whole structure of the network is changed. The new weight matrix must be compatible to the existing connection matrix.

Parameters:

wei

The new 3-dimensional weight matrix, where the first dimension is used for the different time steps with each time step having a 2-dimensional weightMatrix.

Returns:: None.

Definition at line 254 of file MSERNNet.cpp.

References weightMatrix, and writeParameters().

void MSERNNet::setStructure	(	Array< int > &	con,
		Array< double > &	wei
	)

Based on given connection and weight matrices a network is created.

same as setStructure(con); setStructure(wei);#

Parameters:

	con	The 3-dimensional connection matrix, where the first dimension is used for the different time steps with each time step having a 2-dimensional connectionMatrix.
	wei	The new 3-dimensional weight matrix, where the first dimension is used for the different time steps with each time step having a 2-dimensional weightMatrix.

Returns:: None.

Definition at line 248 of file MSERNNet.cpp.

References setStructure().

void MSERNNet::setStructure ( Array< int > & mat )

Based on a given connection matrix a network is created.

This is the core function to set the structure and initialize the sizes of all fields. All other `setStructure' methods call this one. Any old values of the network are lost.

One has to pass a 3-dimensional connection matrix mat to this function. The first dimension indicates the time delay of the respective connection. E.g., mat[0] is the ordinary 2-dimensional feedforward (triangular) connection matrix (where the connections have no time delay), mat[1] is the (potentially fully connected) connection matrix for connections of time delay 1, etc.

It is perfectly ok, if some of these `layers' mat[t] are completely zero: the function will detect this (and store it as a 0 in the delMask(t)) and not waste time for this layer when processing the network.

Notice, that the weight matrix w# of the network is only adapted to the new size, but the matrix is not explicitly initialized with zero weights.

Parameters:

mat

The 3-dimensional connection matrix.

Returns:: None.

Examples:: simpleMSERNNet.cpp.

Definition at line 217 of file MSERNNet.cpp.

References connectionMatrix, delay, delMask, delta, dEw, i, numberOfNeurons, numberOfParameters, Model::parameter, stimulus, weightMatrix, and Y.

Referenced by MSERNNet(), and setStructure().

void MSERNNet::write ( std::string filename )

Writes the structure of the network to a file named "filename".

The structure of the network is written in the following format:

The first line only contains a single value, that denotes the number of time steps used. Following are $number\ of\ time\ steps$ connection matrices, followed by $number\ of\ time\ steps$ weight matrices.

Parameters:

filename

Name of the file, the network structure is written to.

Returns:: None.

Definition at line 324 of file MSERNNet.cpp.

References getConnections(), getWeights(), and i.

void MSERNNet::writeGradient ( Array< double > & derivative ) [protected]

Writes the values of the gradient of the error with respect to the different weights from the variable dEw to the variable dedw in the Model Interface.

Definition at line 441 of file MSERNNet.cpp.

References connectionMatrix, delay, dEw, i, numberOfNeurons, and numberOfParameters.

Referenced by errorDerivative().

void MSERNNet::writeParameters ( ) [protected]

Writes the values of the weights stored in the weight matrix to the parameter vector w of ModelInterface.

Definition at line 421 of file MSERNNet.cpp.

References connectionMatrix, delay, i, numberOfNeurons, Model::parameter, and weightMatrix.

Referenced by initWeights(), and setStructure().

Member Data Documentation

ArrayTable<int> MSERNNet::connectionMatrix [protected]

3-dimensional connection matrix. The first dimension is the time delay, the second and third dimension are the neuron indices of the endpoint and starting point of the connection as given in a "normal" connection matrix.

Definition at line 516 of file MSERNNet.h.

Referenced by calcGradBPTT(), getConnections(), initWeights(), readParameters(), setStructure(), writeGradient(), and writeParameters().

unsigned MSERNNet::delay [protected]

Number of different time steps in the network architecture: delay = 1 means only feed forward, delay > 1 also includes the history of the states into the dynamic.

Definition at line 501 of file MSERNNet.h.

Referenced by calcGradBPTT(), getHistory(), init0(), initWeights(), prepareTime(), processTimeStep(), readParameters(), setHistory(), setStructure(), writeGradient(), and writeParameters().

ArrayTable<int> MSERNNet::delMask [protected]

1-dimensional array with the number of elements equal to the number of time delays in the structure. The i-th element of this array is set to one, if at least one connection from the i-th memory layer to the feed-forward layer exists. This variable is only used to reduce the computational time.

Definition at line 529 of file MSERNNet.h.

Referenced by calcGradBPTT(), processTimeStep(), and setStructure().

ArrayTable<double> MSERNNet::delta [protected]

Stores the local delta values which are used to evaluate the gradient.

Definition at line 550 of file MSERNNet.h.

Referenced by calcGradBPTT(), init0(), prepareTime(), and setStructure().

ArrayTable<double> MSERNNet::dEw [protected]

Derivative of the error with respect to the weights. This object is a 3-dimensional array, the first dimension is the time delay, the second and third dimension are the neuron indices of the endpoint and starting point of the connection, the weight belongs to.

Definition at line 557 of file MSERNNet.h.

Referenced by calcGradBPTT(), setStructure(), and writeGradient().

unsigned MSERNNet::episode [protected]

Definition at line 504 of file MSERNNet.h.

Referenced by calcGradBPTT(), init0(), and prepareTime().

ArrayTable<double> MSERNNet::err [protected]

This array stores the errors of the neurons for every input pattern.

Definition at line 546 of file MSERNNet.h.

Referenced by calcGradBPTT(), error(), errorPercentage(), init0(), and prepareTime().

unsigned MSERNNet::numberOfNeurons [protected]

The total number of neurons of the network (input, output and hidden).

Definition at line 489 of file MSERNNet.h.

Referenced by calcGradBPTT(), error(), errorPercentage(), getHistory(), init0(), initWeights(), model(), prepareTime(), processTimeSeries(), processTimeStep(), readParameters(), setHistory(), setStructure(), writeGradient(), and writeParameters().

unsigned MSERNNet::numberOfParameters [protected]

The absolute number of parameters of the network (equal to the number of weights).

Definition at line 493 of file MSERNNet.h.

Referenced by init0(), setStructure(), and writeGradient().

ArrayTable<double> MSERNNet::stimulus [protected]

Activation of the neurons prior to the processings, usually equal to the input pattern. Stimulus is a 1-dimensional array with the number of elements equal to the number of neurons.

Definition at line 534 of file MSERNNet.h.

Referenced by processTimeSeries(), processTimeStep(), and setStructure().

unsigned MSERNNet::time [protected]

the variable that counts the time (backwards!) when processing a data series

Definition at line 496 of file MSERNNet.h.

Referenced by init0(), prepareTime(), and processTimeStep().

unsigned MSERNNet::warmUpLength [protected]

Number of patterns of the input sequence, which are not considered for evaluating the error and gradient of the error. Nevertheless, these elements are used to modify the internal states of the neurons.

Definition at line 510 of file MSERNNet.h.

Referenced by error(), errorDerivative(), errorPercentage(), includeWarmUp(), init0(), and model().

ArrayTable<double> MSERNNet::weightMatrix [protected]

3-dimensional weight matrix. The first dimension is the time delay, the second and third dimension are the neuron indices of the endpoint and starting point of the connection, the weight belongs to, as given in a "normal" weight matrix.

Definition at line 522 of file MSERNNet.h.

Referenced by calcGradBPTT(), getWeights(), initWeights(), processTimeStep(), readParameters(), setStructure(), and writeParameters().

ArrayTable<double> MSERNNet::Y [protected]

Activation of the neurons after processing the time series. "Y" is a 2-dimensional array, the first dimension gives the neuron index, the second one the time step counted backwards. Therefore, the second dimension's number of elements is equal to the sum of the length of the time series and the maximum time delay of the structure.

Definition at line 542 of file MSERNNet.h.

Referenced by calcGradBPTT(), error(), errorPercentage(), getHistory(), init0(), model(), prepareTime(), processTimeStep(), setHistory(), and setStructure().

The documentation for this class was generated from the following files:


Public Member Functions
	MSERNNet ()
	Creates an empty MSE recurrent neural network.
	MSERNNet (Array< int >)
	Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con".
	MSERNNet (Array< int >, Array< double >)
	Creates an empty MSE recurrent neural network with a certain topology given by the connection matrix "con" and a set of weights given by weight matrix "wei".
	MSERNNet (std::string)
	Creates a recurrent neural network by reading the necessary information from a file named "filename".
void	includeWarmUp (unsigned WUP=0)
	Sets the length of the warmup sequence.
void	model (const Array< double > &input)
	Feed a data series to the model.
void	model (const Array< double > &input, Array< double > &output)
	Feed a data series to the model. The output (i.e., the time series of activations of the output neurons) is copies into the output# buffer.
double	errorPercentage (const Array< double > &input, const Array< double > &target)
	Evaluates the error percentage of the network output compared to given target data.
double	error (Model &model, const Array< double > &input, const Array< double > &target)
	Evaluates the mean squared error of the network output compared to given target data.
double	errorDerivative (Model &model, const Array< double > &input, const Array< double > &target, Array< double > &derivative)
	Evaluates the derivative of the mean squared error.
double	meanSquaredError (const Array< double > &input, const Array< double > &target)
	same as error
Array< int > &	getConnections ()
	Returns the connection Matrix.
Array< double > &	getWeights ()
	Returns the weight matrix.
void	initWeights (double, double)
	Initializes the weights of the net.
void	initWeights (long, double, double)
	Initializes the random number generator and the weights of the net.
void	setStructure (Array< int > &)
	Based on a given connection matrix a network is created.
void	setStructure (Array< int > &, Array< double > &)
	Based on given connection and weight matrices a network is created.
void	setStructure (Array< double > &)
	Replaces the existing weight matrix by a new matrix.
void	setStructure (std::string)
	The current structure of the network is replaced by a new one that is based on the information in file "filename".
void	write (std::string)
	Writes the structure of the network to a file named "filename".
void	setStructur (unsigned in, unsigned hidden, unsigned out, unsigned memory=1, bool layered=false, bool recurrentInputs=true, bool bias=true, bool elman=false, bool previousInputs=false)
	Creates a connection matrix for a recurrent network.
void	setHistory (const Array< double > &)
	Sets the history of the neuron states to certain values.
Array< double >	getHistory ()
	Returns the history of the neuron states.
Protected Member Functions
virtual double	g (double)
	Activation function of all neurons.
virtual double	dg (double)
	Computes the derivative of g(a) for all neurons.
void	init0 ()
	Initializes some internal variables.
void	writeParameters ()
	Writes the values of the weights stored in the weight matrix to the parameter vector w of ModelInterface.
void	readParameters ()
	Reads the values of the parameter vector w of ModelInterface and stores these values in the weight matrix.
void	writeGradient (Array< double > &derivative)
void	prepareTime (unsigned t)
	Performs some initializations that are necessary to process the time series.
void	processTimeSeries (const Array< double > &input)
	Processes a whole time series.
void	processTimeStep ()
	Processes one input pattern.
void	calcGradBPTT ()
Protected Attributes
unsigned	numberOfNeurons
	The total number of neurons of the network (input, output and hidden).
unsigned	numberOfParameters
	The absolute number of parameters of the network (equal to the number of weights).
unsigned	time
	the variable that counts the time (backwards!) when processing a data series
unsigned	delay
unsigned	episode
unsigned	warmUpLength
ArrayTable< int >	connectionMatrix
ArrayTable< double >	weightMatrix
ArrayTable< int >	delMask
ArrayTable< double >	stimulus
ArrayTable< double >	Y
ArrayTable< double >	err
	This array stores the errors of the neurons for every input pattern.
ArrayTable< double >	delta
	Stores the local delta values which are used to evaluate the gradient.
ArrayTable< double >	dEw