SVM Class Reference

Support Vector Machine (SVM) as a ReClaM Model. More...

#include <Svm.h>

Inheritance diagram for SVM:

List of all members.

Public Member Functions

SVM (KernelFunction *pKernel, bool bSignOutput=false)

Constructor.

SVM (KernelFunction *pKernel, const Array< double > &input, bool bSignOutput=false)

Constructor.

~SVM ()

Destructor.

void SetTrainingData (const Array< double > &input, bool copy=false)

void model (const Array< double > &input, Array< double > &output)

compute the SVM prediction on data

double model (const Array< double > &input)

compute the SVM prediction on data

void modelDerivative (const Array< double > &input, Array< double > &derivative)

void modelDerivative (const Array< double > &input, Array< double > &output, Array< double > &derivative)

double getAlpha (int index)

double getOffset ()

KernelFunction * getKernel ()

return the kernel function object

const Array< double > & getPoints ()

return the training data points

unsigned int getExamples ()

return the number of training examples

unsigned int getDimension ()

return the input space dimension

void MakeSparse ()

Discard zero coefficients and corresponding training data.

bool LoadSVMModel (std::istream &is)

bool SaveSVMModel (std::ostream &os)

Static Public Member Functions

static SVM * ImportLibsvmModel (std::istream &is)

static SVM * ImportSvmlightModel (std::istream &is)

Static Protected Member Functions

static int ReadToken (std::istream &is, char *buffer, int maxlength, const char *separators)

Read a token from a stream.

static int DiscardUntil (std::istream &is, const char *separators)

Discard characters with stopping condition.

Protected Attributes

KernelFunction * kernel

kernel function

bool bOwnMemory

true if x and y are allocated by LoadModel

const Array< double > * x

training data points

bool signOutput

true if the SVM outputs the binary label $\pm 1$ only, false if the SVM outputs the value of the linear feature space function.

unsigned int examples

number of training data points x and labels y

Friends

class SvmApproximation

Detailed Description

Support Vector Machine (SVM) as a ReClaM Model.

Author:: T. Glasmachers

Date:: 2005

The SVM class provides a Support Vector Machine as a: parametric Model, that is, it computes a linear expansion of the kernel function with fixed training examples as one component. It can be viewed as a parametrized family of maps from the input space to the reals.

The parameter array of the SVM class defines the: affine linear solution in feature space, usually described as a vector $\alpha$ and a real valued offset b. Note that different SVM training procedures impose constraints on the possible values these parameters are allowed to take.

In ReClaM, the SVM as a model is used for prediction: only. That is, it does not impose any training scheme and could in theory be training using any error measure and any optimizer. In practice, one wants to apply standard SVM training schemes to efficiently find the SVM solution. Therefore, the SVM should be trained with the special optimizer derived class C_SVM. This class implements the so-called C-SVM as a training scheme and uses a quadratic program solver to obtain the optimal solution. However, some other training schemes like basic regularization networks and gaussian process work on the same type of model. Therefore the SVM should be understood as an affine linear model in the kernel induced feature space, rather than being too closely connected to standard SVM training schemes.

Examples:: CrossValidation.cpp, KernelOptimization.cpp, SvmApproximationExample.cpp, SVMclassification-gnuplot.cpp, SVMclassification.cpp, SVMregression-gnuplot.cpp, and SVMregression.cpp.

Definition at line 88 of file Svm.h.

Constructor & Destructor Documentation

SVM::SVM	(	KernelFunction *	pKernel,
		bool	bSignOutput = `false`
	)

Constructor.

Parameters:

	pKernel	kernel function to use for training and prediction
	bSignOutput	true if the SVM should output binary labels, false if it should output real valued function evaluations

Definition at line 57 of file Svm.cpp.

References bOwnMemory, examples, Model::inputDimension, kernel, Model::outputDimension, Model::parameter, signOutput, and x.

Referenced by ImportLibsvmModel(), and ImportSvmlightModel().

SVM::SVM	(	KernelFunction *	pKernel,
		const Array< double > &	input,
		bool	bSignOutput = `false`
	)

Constructor.

Parameters:

	pKernel	kernel function to use for training and prediction
	input	training data points
	bSignOutput	true if the SVM should output binary labels, false if it should output real valued function evaluations

Definition at line 73 of file Svm.cpp.

References bOwnMemory, examples, Model::inputDimension, kernel, Model::outputDimension, SetTrainingData(), signOutput, and x.

SVM::~SVM ( )

Destructor.

Definition at line 88 of file Svm.cpp.

References bOwnMemory, and x.

Member Function Documentation

int SVM::DiscardUntil	(	std::istream &	is,
		const char *	separators
	)			`[static, protected]`

Discard characters with stopping condition.

: The function reads characters from a stream until one of the terminating characters is found.

Parameters:

	is	stream to read from
	separators	termination characters

Returns:

A status code is returned:

terminating char
end of file --> 1001
unrecoverable error --> 1002

Definition at line 685 of file Svm.cpp.

Referenced by ImportSvmlightModel().

double SVM::getAlpha ( int index ) [inline]

: retrieve one of the coeffitients from the solution vector usually referred to as $\alpha$

Author:: T. Glasmachers

Date:: 2006

Examples:: SVMclassification-gnuplot.cpp.

Definition at line 180 of file Svm.h.

References Model::parameter.

Referenced by SvmApproximation::addVecFixPointIteration(), SvmApproximation::calcOffsetForReducedModel(), SvmApproximation::calcOptimalAlphaOfApproximatedSVM(), SvmApproximation::chooseVectorForNextIteration(), SvmApproximation::createIndexListWithKMeans(), SvmApproximation::createIndexListWithStochasticUniversalSampling(), SvmApproximation::error(), SpanBound1::error(), SpanBound::error(), SvmApproximationErrorFunctionGlobal::errorDerivative(), SvmApproximationErrorFunction::errorDerivative(), SpanBound1::errorDerivative(), SvmApproximation::getClassificationRateOnSVsCorrectlyClassifiedByOrigSVM(), SvmApproximation::gradientDescent(), C_SVM::modelDerivative(), C_SVM::PrepareDerivative(), and SvmApproximation::SvmApproximation().

unsigned int SVM::getDimension ( ) [inline]

return the input space dimension

Definition at line 216 of file Svm.h.

References Model::inputDimension.

Referenced by SvmApproximationErrorFunctionGlobal::errorDerivative(), SvmApproximationErrorFunction::errorDerivative(), and SvmApproximation::SvmApproximation().

unsigned int SVM::getExamples ( ) [inline]

return the number of training examples

Definition at line 210 of file Svm.h.

References examples.

Referenced by GaussianProcessEvidence::dEvidenceDTheta(), SpanBound1::error(), GaussianProcessVariance::error(), SvmApproximationErrorFunctionGlobal::errorDerivative(), SvmApproximationErrorFunction::errorDerivative(), SpanBound1::errorDerivative(), GaussianProcessEvidence::evidence(), MakeSparse(), C_SVM::modelDerivative(), C_SVM::PrepareDerivative(), and SvmApproximation::SvmApproximation().

KernelFunction* SVM::getKernel ( ) [inline]

double SVM::getOffset ( ) [inline]

: retrieve the solution offset, usually referred to as

Author:: T. Glasmachers

Date:: 2006

Examples:: SVMclassification-gnuplot.cpp.

Definition at line 192 of file Svm.h.

References examples, and Model::parameter.

Referenced by SvmApproximation::calcOffsetForReducedModel(), and C_SVM::PrepareDerivative().

const Array<double>& SVM::getPoints ( ) [inline]

return the training data points

Definition at line 204 of file Svm.h.

References x.

Referenced by SvmApproximation::addVecFixPointIteration(), SvmApproximation::calcOffsetForReducedModel(), SvmApproximation::calcOptimalAlphaOfApproximatedSVM(), SvmApproximation::chooseVectorForNextIteration(), SvmApproximation::createIndexListWithKMeans(), GaussianProcessEvidence::dEvidenceDTheta(), SvmApproximation::error(), GaussianProcessVariance::error(), SvmApproximationErrorFunctionGlobal::errorDerivative(), SvmApproximationErrorFunction::errorDerivative(), SvmApproximation::getClassificationRateOnSVsCorrectlyClassifiedByOrigSVM(), C_SVM::modelDerivative(), and C_SVM::PrepareDerivative().

SVM * SVM::ImportLibsvmModel ( std::istream & is ) [static]

: Import a libsvm 2.81 model file.

Author:: T. Glasmachers

Date:: 2006

Returns:: On success a SVM model is returned. A return value of NULL indicates an error.

Definition at line 346 of file Svm.cpp.

References bOwnMemory, examples, i, Model::inputDimension, Model::parameter, ReadToken(), Model::setParameter(), PolynomialKernel::setParameter(), SVM(), and x.

SVM * SVM::ImportSvmlightModel ( std::istream & is ) [static]

: Import an SVM-light model file.

Author:: T. Glasmachers

Date:: 2006

Returns:: On success a SVM model is returned. A return value of NULL indicates an error.

Definition at line 503 of file Svm.cpp.

References bOwnMemory, DiscardUntil(), examples, i, Model::inputDimension, kernel, Model::parameter, ReadToken(), SVM(), and x.

bool SVM::LoadSVMModel ( std::istream & is )

: Load the complete SVM model including kernel parameters, alpha, b and the support vectors.

Author:: T. Glasmachers

Date:: 2006

Definition at line 221 of file Svm.cpp.

References bOwnMemory, examples, Model::inputDimension, Model::parameter, and x.

void SVM::MakeSparse ( )

Discard zero coefficients and corresponding training data.

: If the SVM object onws its training data array, it can make itself sparse after training.

Definition at line 627 of file Svm.cpp.

References bOwnMemory, examples, getExamples(), i, Model::parameter, and x.

Referenced by SVM_Optimizer::optimize().

double SVM::model ( const Array< double > & input )

compute the SVM prediction on data

Author:: T. Glasmachers

Date:: 2006

Definition at line 159 of file Svm.cpp.

References KernelFunction::eval(), examples, i, kernel, Model::parameter, signOutput, and x.

void SVM::model	(	const Array< double > &	input,
		Array< double > &	output
	)			`[virtual]`

compute the SVM prediction on data

Author:: T. Glasmachers

Date:: 2006

Implements Model.

Definition at line 116 of file Svm.cpp.

References KernelFunction::eval(), examples, i, kernel, Model::parameter, signOutput, and x.

Referenced by SpanBound1::error(), SpanBound::error(), NoisySvmLikelihood::error(), SpanBound1::errorDerivative(), NoisySvmLikelihood::errorDerivative(), SvmApproximation::getClassificationRateOnSVsCorrectlyClassifiedByOrigSVM(), GaussianProcess::operator()(), and Perceptron::optimize().

void SVM::modelDerivative	(	const Array< double > &	input,
		Array< double > &	output,
		Array< double > &	derivative
	)			`[virtual]`

: The modelDerivative member computes the derivative of the SVM function w.r.t. its parameters. Although this information is probably never used, it is easy to compute :)

Of course, the derivative does not make sense after the application of the sign function. Thus, this method always returns the derivative w.r.t. the real valued SVM output.

Author:: T. Glasmachers

Date:: 2006

Reimplemented from Model.

Definition at line 199 of file Svm.cpp.

References KernelFunction::eval(), examples, i, kernel, Model::parameter, and x.

void SVM::modelDerivative	(	const Array< double > &	input,
		Array< double > &	derivative
	)			`[virtual]`

: The modelDerivative member computes the derivative of the SVM function w.r.t. its parameters. Although this information is probably never used, it is easy to compute :)

Of course, the derivative does not make sense after the application of the sign function. Thus, this method always returns the derivative w.r.t. the real valued SVM output.

Author:: T. Glasmachers

Date:: 2006

Reimplemented from Model.

Definition at line 181 of file Svm.cpp.

References KernelFunction::eval(), examples, i, kernel, and x.

int SVM::ReadToken	(	std::istream &	is,
		char *	buffer,
		int	maxlength,
		const char *	separators
	)			`[static, protected]`

Read a token from a stream.

: The function reads a token from a stream. It stores the token into a null terminated string. If the first character read is already a separator the function returns an empty token.

Parameters:

	is	stream to read from
	buffer	buffer to store the token
	maxlength	size of the buffer
	separators	characters terminating the token

Returns:

A status code is returned:

token read successfully --> terminating char
token read successfully, end of file --> 1001
unrecoverable error --> 1002
buffer overrun, token truncated to size maxlength-1 --> 1003

Definition at line 663 of file Svm.cpp.

References i.

Referenced by ImportLibsvmModel(), and ImportSvmlightModel().

bool SVM::SaveSVMModel ( std::ostream & os )

: Save the complete SVM model including kernel parameters, alpha, b and the support vectors.

Author:: T. Glasmachers

Date:: 2006

Definition at line 295 of file Svm.cpp.

References examples, Model::inputDimension, and Model::parameter.

void SVM::SetTrainingData	(	const Array< double > &	input,
		bool	copy = `false`
	)

: As the SVM can be constructed without training data points, although these are needed for the computation of the model, this member makes the training data known to the SVM. This method is usually called by the SVM_Optimizer class, but in case a model was loaded from a file it can be necessary to invoke the function manually.

: A side effects, the method eventually rescales the parameter vector. In any case, all parameters are reset. The number of examples and the input space dimension are overwritten.

Author:: T. Glasmachers

Date:: 2006

Parameters:

	input	training data points
	copy	maintain a copy of the input data

Definition at line 94 of file Svm.cpp.

References bOwnMemory, examples, Model::inputDimension, Model::parameter, and x.

Referenced by LOO::error(), SVM_Optimizer::optimize(), Perceptron::optimize(), SVM(), SvmApproximation::SvmApproximation(), and GaussianProcess::train().

Friends And Related Function Documentation

friend class SvmApproximation [friend]

Definition at line 272 of file Svm.h.

Member Data Documentation

bool SVM::bOwnMemory [protected]

true if x and y are allocated by LoadModel

Definition at line 318 of file Svm.h.

Referenced by ImportLibsvmModel(), ImportSvmlightModel(), LoadSVMModel(), MakeSparse(), SetTrainingData(), SVM(), and ~SVM().

unsigned int SVM::examples [protected]

number of training data points x and labels y

Definition at line 328 of file Svm.h.

Referenced by getExamples(), getOffset(), ImportLibsvmModel(), ImportSvmlightModel(), LoadSVMModel(), MakeSparse(), model(), modelDerivative(), SaveSVMModel(), SetTrainingData(), SVM(), and SvmApproximation::SvmApproximation().

KernelFunction* SVM::kernel [protected]

kernel function

Definition at line 315 of file Svm.h.

Referenced by getKernel(), ImportSvmlightModel(), model(), modelDerivative(), and SVM().

bool SVM::signOutput [protected]

true if the SVM outputs the binary label $\pm 1$ only, false if the SVM outputs the value of the linear feature space function.

Definition at line 325 of file Svm.h.

Referenced by model(), and SVM().

const Array<double>* SVM::x [protected]

training data points

Definition at line 321 of file Svm.h.

Referenced by getPoints(), ImportLibsvmModel(), ImportSvmlightModel(), LoadSVMModel(), MakeSparse(), model(), modelDerivative(), SetTrainingData(), SVM(), and ~SVM().

The documentation for this class was generated from the following files:


Public Member Functions
	SVM (KernelFunction *pKernel, bool bSignOutput=false)
	Constructor.
	SVM (KernelFunction *pKernel, const Array< double > &input, bool bSignOutput=false)
	Constructor.
	~SVM ()
	Destructor.
void	SetTrainingData (const Array< double > &input, bool copy=false)
void	model (const Array< double > &input, Array< double > &output)
	compute the SVM prediction on data
double	model (const Array< double > &input)
	compute the SVM prediction on data
void	modelDerivative (const Array< double > &input, Array< double > &derivative)
void	modelDerivative (const Array< double > &input, Array< double > &output, Array< double > &derivative)
double	getAlpha (int index)
double	getOffset ()
KernelFunction *	getKernel ()
	return the kernel function object
const Array< double > &	getPoints ()
	return the training data points
unsigned int	getExamples ()
	return the number of training examples
unsigned int	getDimension ()
	return the input space dimension
void	MakeSparse ()
	Discard zero coefficients and corresponding training data.
bool	LoadSVMModel (std::istream &is)
bool	SaveSVMModel (std::ostream &os)
Static Public Member Functions
static SVM *	ImportLibsvmModel (std::istream &is)
static SVM *	ImportSvmlightModel (std::istream &is)
Static Protected Member Functions
static int	ReadToken (std::istream &is, char buffer, int maxlength, const char separators)
	Read a token from a stream.
static int	DiscardUntil (std::istream &is, const char *separators)
	Discard characters with stopping condition.
Protected Attributes
KernelFunction *	kernel
	kernel function
bool	bOwnMemory
	true if x and y are allocated by LoadModel
const Array< double > *	x
	training data points
bool	signOutput
	true if the SVM outputs the binary label $\pm 1$ only, false if the SVM outputs the value of the linear feature space function.
unsigned int	examples
	number of training data points x and labels y
Friends
class	SvmApproximation