DF_CrossEntropyIndependent Class Reference

Error measure for classification tasks of non exclusive attributes that can be used for model training. More...

#include <DF_CrossEntropyIndependent.h>

Inheritance diagram for DF_CrossEntropyIndependent:

ErrorFunction

List of all members.

Public Member Functions

double error (Model &model, const Array< double > &in, const Array< double > &target)
 Calculates the CrossEntropyIndependent error.
double errorDerivative (Model &model, const Array< double > &input, const Array< double > &target, Array< double > &derivative)
 Calculates the derivatives of the CrossEntropyIndependent error (see error) with respect to the parameters ErrorFunction::w.

Detailed Description

Error measure for classification tasks of non exclusive attributes that can be used for model training.

If your model should return a vector whose components are connected to multiple mutually independent attributes and the k-th component of the output vector is representing the probability of the presence of the k-th attribute given any input vector, 'DF_CrossEntropyIndependent' is the adequate error measure for model-training. For C>1, dimension of model's output and every output dimension represents a single attribute or class respectively, it follows the formular

\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{target^i_k \ln model_k(input^i) + (1-target^i_k) \ln (1-model_k(input^i))\} \]

where i runs over all input patterns. This error functional can be derivated and so used for training. In case of only one single output dimension 'DF_CrossEntropyIndependent' returns actually the true cross entropy for two classes, using the formalism

\[ E = - \sum_{i=1}^N \{target^i \ln model(input^i) + (1-target^i) \ln (1-model(input^i))\} \]

For theoretical reasons it is suggested to use for neural networks the logistic sigmoid activation function at the output neurons. However, actually every sigmoid activation could be applied to the output neurons as far as the image of this function is identical to the intervall [0,1]. In this implementation every target value to be chosen from {0,1} (binary encoding). For detailed information refer to (C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press 1996, Chapter 6.8.)

This implementation of 'CrossEntropyIndependent' performs more efficient than the implemetation given in 'CrossEntropyIndependent.h' for more than one output dimensions, because redundant calculations of the outer derivatives are circumvented.

Author:
S. Wiegand
Date:
2003
Changes:
none
Status:
stable

Definition at line 129 of file DF_CrossEntropyIndependent.h.

Member Function Documentation

double DF_CrossEntropyIndependent::error ( Model &  model,
const Array< double > &  in,
const Array< double > &  target 
) [inline, virtual]

Calculates the CrossEntropyIndependent error.

Calculates the CrossEntropyIndependent error function for N patterns and C>1 independent attributes within the output vector via

\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{target^i_k \ln model_k(input^i) + (1-target^i_k) \ln (1-model_k(input^i))\} \]

or the true cross entropy for only one single output dimension via

\[ E = - \sum_{i=1}^N \{target^i \ln model(input^i) + (1-target^i) \ln (1-model(input^i))\} \]

Parameters:
model the model.
input Input vector for the model.
target Target vector.
Returns:
The error E.
Author:
S. Wiegand
Date:
2003
Changes
none
Status
stable

Implements ErrorFunction.

Definition at line 167 of file DF_CrossEntropyIndependent.h.

References Model::getOutputDimension(), and Model::model().

double DF_CrossEntropyIndependent::errorDerivative ( Model &  model,
const Array< double > &  input,
const Array< double > &  target,
Array< double > &  derivative 
) [inline, virtual]

Calculates the derivatives of the CrossEntropyIndependent error (see error) with respect to the parameters ErrorFunction::w.

Calculates the CrossEntropyIndependent error derivative for N patterns and C>1 independent attributes within the output vector corresponding to model parameter w via

\[ \frac{\partial E}{\partial w} = - \sum_{i=1}^N \sum_{k=1}^{C} \left\{ \frac{target^i_k - model_k(input^i)}{model_k(input^i)\cdot(1-model_k(input^i))} \cdot\frac{\partial model_k(input^i)}{\partial w}\right\} \]

or the derivative of the true cross entropy for only one single output dimension via

\[ \frac{\partial E}{\partial w} = - \sum_{i=1}^N \left\{ \frac{target^i - model(input^i)}{model(input^i)\cdot(1-model(input^i))} \cdot\frac{\partial model(input^i)}{\partial w}\right\} \]

Usually, as a byproduct of the calculation of the derivative one gets the CrossEntropyIndependent error itself very efficiently. Therefore, the method error gives back this value. This additional effect can be switched of by means of the third parameter (returnError = false).

Parameters:
model the model
input Input vector for the model.
target Target vector.
derivative the error derrivative
Returns:
The CrossEntropyIndependent error
Author:
S. Wiegand
Date:
2003
Changes
none
Status
stable

Reimplemented from ErrorFunction.

Definition at line 266 of file DF_CrossEntropyIndependent.h.

References Model::generalDerivative(), Model::getOutputDimension(), and Model::model().

The documentation for this class was generated from the following file: