CrossEntropyIndependent Class Reference

Error measure for classification tasks of non exclusive attributes that can be used for model training. More...

#include <CrossEntropyIndependent.h>

Inheritance diagram for CrossEntropyIndependent:
ErrorFunction

List of all members.

Public Member Functions

double error (Model &model, const Array< double > &in, const Array< double > &target)
 Calculates the CrossEntropyIndependent error.
double errorDerivative (Model &model, const Array< double > &in, const Array< double > &target, Array< double > &derivative)
 Calculates the derivatives of the CrossEntropyIndependent error (see error) with respect to the parameters ModelInterface::w.

Detailed Description

Error measure for classification tasks of non exclusive attributes that can be used for model training.

If your model should return a vector whose components are connected to multiple mutually independent attributes and the k-th component of the output vector is representing the probability of the presence of the k-th attribute given any input vector, 'CrossEntropyIndependent' is the adequate error measure for model-training. For C>1, dimension of model's output and every output dimension represents a single attribute or class respectively, it follows the formular

\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{tar^i_k \ln model_k(in^i) + (1-tar^i_k) \ln (1-model_k(in^i))\} \]

where i runs over all input patterns. This error functional can be derivated and so used for training. In case of only one single output dimension 'CrossEntropyIndependent' returns actually the true cross entropy for two classes, using the formalism

\[ E = - \sum_{i=1}^N \{tar^i \ln model(in^i) + (1-tar^i) \ln (1-model(in^i))\} \]

For theoretical reasons it is suggested to use for neural networks the logistic sigmoid activation function at the output neurons. However, actually every sigmoid activation could be applied to the output neurons as far as the image of this function is identical to the intervall [0,1]. In this implementation every target value to be chosen from {0,1} (binary encoding). For detailed information refer to (C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press 1996, Chapter 6.8.)

This implementation of the cross entropy performs less efficient than the implemetation given in 'DF_CrossEntropyIndependent.h' for more than one output dimensions, if standard model classes as 'FFNet' are simultaneously linked, because redundant calculations of the outer derivatives are not circumvented.

Author:
M. Huesken
Date:
1999
Changes:
Revision 2003/06/03 (S. Wiegand): bugs in formulars corrected
Status:
stable

Definition at line 131 of file CrossEntropyIndependent.h.

Member Function Documentation

double CrossEntropyIndependent::error ( Model &  model,
const Array< double > &  in,
const Array< double > &  target 
) [inline, virtual]

Calculates the CrossEntropyIndependent error.

Calculates the CrossEntropyIndependent error function for N patterns and C>1 independent attributes within the output vector via

\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{tar^i_k \ln model_k(in^i) + (1-tar^i_k) \ln (1-model_k(in^i))\} \]

or the true cross entropy for only one single output dimension via

\[ E = - \sum_{i=1}^N \{tar^i \ln model(in^i) + (1-tar^i) \ln (1-model(in^i))\} \]

Parameters:
in Input vector for the model.
target Target vector.
Returns:
The error E.
Author:
M. Huesken, S. Wiegand
Date:
1999, 2003

Implements ErrorFunction.

Definition at line 162 of file CrossEntropyIndependent.h.

References Model::getOutputDimension().

double CrossEntropyIndependent::errorDerivative ( Model &  model,
const Array< double > &  in,
const Array< double > &  target,
Array< double > &  derivative 
) [inline, virtual]

Calculates the derivatives of the CrossEntropyIndependent error (see error) with respect to the parameters ModelInterface::w.

Calculates the CrossEntropyIndependent error derivative for N patterns and C>1 independent attributes within the output vector corresponding to model parameter w via

\[ \frac{\partial E}{\partial w} = - \sum_{i=1}^N \sum_{k=1}^{C} \left\{ \frac{tar^i_k - model_k(in^i)}{model_k(in^i)\cdot(1-model_k(in^i))} \cdot\frac{\partial model_k(in^i)}{\partial w}\right\} \]

or the derivative of the true cross entropy for only one single output dimension via

\[ \frac{\partial E}{\partial w} = - \sum_{i=1}^N \left\{ \frac{tar^i - model(in^i)}{model(in^i)\cdot(1-model(in^i))} \cdot\frac{\partial model(in^i)}{\partial w}\right\} \]

Usually, as a byproduct of the calculation of the derivative one gets the CrossEntropyIndependent error itself very efficiently. Therefore, the method error gives back this value. This additional effect can be switched of by means of the third parameter (returnError = false).

Parameters:
in Input vector for the model.
target Target vector.
returnError Determines whether or not to calculate the cross entropy error itself. By default the error is calculated.
Returns:
The CrossEntropyIndependent error if returnError is set to "true", "-1" otherwise.
Author:
M. Huesken
Date:
1999
Changes
Revision 2003/06/03 (S. Wiegand): bugs in formulars corrected
Status
stable

Reimplemented from ErrorFunction.

Definition at line 263 of file CrossEntropyIndependent.h.

References Model::getOutputDimension(), Model::getParameterDimension(), i, and Model::modelDerivative().

The documentation for this class was generated from the following file: