ErrorFunction Class Reference

Base class of all error measures. More...

#include <ErrorFunction.h>

Inheritance diagram for ErrorFunction:
BalancedClassificationError BinaryCriterion ClassificationError CrossEntropy CrossEntropyIndependent CVError DF_CrossEntropy DF_CrossEntropyIndependent DF_MeanSquaredError ErrorPercentage GaussianProcessEvidence GaussianProcessVariance InverseClassSeparability LinearEquation LOO MeanSquaredError ModelInterface ModelWithErrorFunction MSEFFNet MSERNNet NegativeBKTA NegativeKTA NegativeLogLikelihood NegativePolarization NoisySvmLikelihood Paraboloid RadiusMargin SpanBound SpanBound1 SquaredError SvmApproximationErrorFunction SvmApproximationErrorFunctionGlobal ValidationError WTA ZeroOneLoss

List of all members.

Public Member Functions

 ErrorFunction ()
 Constructor.
virtual ~ErrorFunction ()
 Destructor.
virtual double error (Model &model, const Array< double > &input, const Array< double > &target)=0
 Error of the model.
virtual double errorDerivative (Model &model, const Array< double > &input, const Array< double > &target, Array< double > &derivative)
 Calculates the field derivative, the derivative of the error $E$ with respect to the model parameters.
void setEpsilon (double eps)
 Sets the value of epsilon.

Protected Attributes

double epsilon
 Precision parameter for the numerical error gradient approximation.

Detailed Description

Base class of all error measures.

ReClaM provides the three base classes Model, ErrorFunction and Optimizer which make up the ReClaM framework for solving regression and classification tasks. This design overrides the ModelInterface design which is kept for downward compatibility.
The ErrorFunction class encapsulates an error function operating on any model defined by the Model class.

An ErrorFunction computes a risk, that is, a loss function over a set of training patterns. It takes a model, a set of input patterns and a set of corresponding targets as its input, and outputs a single double value (the error).

The input array can be expected to be two dimensional, indexed by the patterns and the input space dimension. In the general case, the target will be two dimensional, too, indexed by the patterns and the model output dimension. However, ErrorFunctions should be aware of the case that the underlying Model is a function. In this case model output and target array may lack the output dimension, that is, they may be one dimensional, indexed by the patterns. It possible (and meaningful), ErrorFunctions should handle both cases. If not, they should throw an exception in the unhandled case.

Author:
T. Glasmachers, C. Igel
Date:
2005

Definition at line 81 of file ErrorFunction.h.

Constructor & Destructor Documentation

ErrorFunction::ErrorFunction (  ) 

Constructor.

Definition at line 70 of file ErrorFunction.cpp.

References epsilon.

ErrorFunction::~ErrorFunction (  )  [virtual]

Destructor.

Definition at line 75 of file ErrorFunction.cpp.

Member Function Documentation

virtual double ErrorFunction::error ( Model &  model,
const Array< double > &  input,
const Array< double > &  target 
) [pure virtual]

Error of the model.

This method calculates the error of the model. target is compared with the model's output for stimulus input.

Parameters:
model Model to use for the computation.
input Vector of input values.
target Vector of output values.
Returns:
The error of the model.
Author:
C. Igel
Date:
1999
Changes
none
Status
stable

Implemented in BinaryCriterion, ClassificationError, BalancedClassificationError, ZeroOneLoss, CrossEntropy, CrossEntropyIndependent, CVError, DF_CrossEntropy, DF_CrossEntropyIndependent, DF_MeanSquaredError, ErrorPercentage, GaussianProcessVariance, GaussianProcessEvidence, InverseClassSeparability, NegativeKTA, NegativeBKTA, LinearEquation, LMSEFFNet, LOO, MeanSquaredError, MSEFFNet, MSERNNet, NegativeLogLikelihood, NegativePolarization, NoisySvmLikelihood, Paraboloid, RadiusMargin, SpanBound, SpanBound1, SquaredError, SvmApproximationErrorFunction, SvmApproximationErrorFunctionGlobal, ValidationError, and WTA.

Referenced by ValidationError::error(), CVError::error(), errorDerivative(), CMAOptimizer::ModelFitness::fitness(), PointSearch::optimize(), NestedGridSearch::optimize(), GridSearch::optimize(), and CG::optimize().

double ErrorFunction::errorDerivative ( Model &  model,
const Array< double > &  input,
const Array< double > &  target,
Array< double > &  derivative 
) [virtual]

Calculates the field derivative, the derivative of the error $E$ with respect to the model parameters.

Calculates the derivates of the error $E$ with respect to the model parameters and stores the result in the elements of the array derivative. As a default implementation, the derivatives are estimated numerically by using small variations epsilon:

\[\frac{\partial E}{\partial \omega_i} = \frac{E(\omega_i + \epsilon) - E(\omega_i)}{\epsilon} + O(\epsilon) \]

However, to improve the speed, it is adviserable to overload this method by a more sophisticated calculation, depending on the method error. Nevertheless, this default implementation can be used to check new implementations of the derivative of the error with respect to the model parameters.

Usually, as a byproduct of the calculation of the derivative one gets the the error $E$ itself very efficiently. Therefore, the method errorDerivative gives back this value.

Parameters:
model Model to use for the computation.
input Vector of input values.
target Vector of output values.
derivative Vector of partial derivatives.
Returns:
The error E
Author:
C. Igel
Date:
1999
Changes
3/2006 Tobias Glasmachers: step size epsilon has bacome a fraction of the current parameter value
Status
stable

Reimplemented in CrossEntropy, CrossEntropyIndependent, CVError, DF_CrossEntropy, DF_CrossEntropyIndependent, DF_MeanSquaredError, GaussianProcessEvidence, InverseClassSeparability, NegativeKTA, NegativeBKTA, LinearEquation, LMSEFFNet, MeanSquaredError, MSEFFNet, MSERNNet, NegativeLogLikelihood, NegativePolarization, NoisySvmLikelihood, Paraboloid, RadiusMargin, SpanBound1, SquaredError, SvmApproximationErrorFunction, SvmApproximationErrorFunctionGlobal, and ValidationError.

Definition at line 80 of file ErrorFunction.cpp.

References epsilon, error(), Model::getParameter(), Model::getParameterDimension(), and Model::setParameter().

Referenced by BFGS::bfgs(), ValidationError::errorDerivative(), CVError::errorDerivative(), StochasticGradientDescent::optimize(), SteepestDescent::optimize(), IRpropMinus::optimize(), IRpropPlus::optimize(), RpropMinus::optimize(), RpropPlus::optimize(), QuickpropOriginal::optimize(), Quickprop::optimize(), NoisyRprop::optimize(), CG::optimize(), AdpBP90b::optimize(), and AdpBP90a::optimize().

void ErrorFunction::setEpsilon ( double  eps  )  [inline]

Sets the value of epsilon.

The value of epsilon is utilized for a numeric estimation of the derivative of the error, with respect to the model parameters. epsilon should be choosen as a small value. The default value is epsilon = 1e-8.

Parameters:
eps New value for epsilon.
Returns:
None.
Author:
C. Igel
Date:
1999
Changes
none
Status
stable

Definition at line 182 of file ErrorFunction.h.

References epsilon.

Member Data Documentation

double ErrorFunction::epsilon [protected]

Precision parameter for the numerical error gradient approximation.

Definition at line 185 of file ErrorFunction.h.

Referenced by errorDerivative(), ErrorFunction(), and setEpsilon().

The documentation for this class was generated from the following files: