BFGS.cpp File Reference

Offers the Broyden-Fletcher-Goldfarb-Shanno algorithm for the optimization of models. More...

#include <cmath>
#include <iostream>
#include <float.h>
#include <SharkDefs.h>
#include <ReClaM/BFGS.h>

Go to the source code of this file.

Defines

#define DLINMIN
#define SIGN(a, b)   ((b) >= 0.0 ? fabs(a) : -fabs(a))
 Just used to override the standard definition.

Variables

unsigned i
 Initializes some internal variables used by the BFGS algorithm, calculates the current error of the model.

Detailed Description

Offers the Broyden-Fletcher-Goldfarb-Shanno algorithm for the optimization of models.

Author:
C. Igel
Date:
1999
Copyright (c) 1999
Institut für Neuroinformatik
Ruhr-Universität Bochum
D-44780 Bochum, Germany
Phone: +49-234-32-25558
Fax: +49-234-32-14209
eMail: Shark-admin@neuroinformatik.ruhr-uni-bochum.de
www: http://www.neuroinformatik.ruhr-uni-bochum.de

Project:
ReClaM


This file is part of ReClaM. This library is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this library; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

---------------------------------------------------------------------

This code is a modification of the original code from the library:

============================================================ COOOL version 1.1 --- Nov, 1995 Center for Wave Phenomena, Colorado School of Mines ============================================================

This code is part of a preliminary release of COOOL (CWP Object-Oriented Optimization Library) and associated class libraries.

The COOOL library is a free software. You can do anything you want with it including make a fortune. However, neither the authors, the Center for Wave Phenomena, nor anyone else you can think of makes any guarantees about anything in this package or any aspect of its functionality.

Since you've got the source code you can also modify the library to suit your own purposes. We would appreciate it if the headers that identify the authors are kept in the source code.

================================== author: H. Lydia Deng, 06/17/96 ==================================

Definition in file BFGS.cpp.

Define Documentation

#define DLINMIN

Definition at line 83 of file BFGS.cpp.

#define SIGN ( a,
b   )     ((b) >= 0.0 ? fabs(a) : -fabs(a))

Just used to override the standard definition.

Definition at line 494 of file BFGS.cpp.

Variable Documentation

unsigned i

Initializes some internal variables used by the BFGS algorithm, calculates the current error of the model.

The given parameters are used to initialize some variables used by the BFGS and the line search algorithms and to allocate memory for internal structures. The current error value of the used model is determined.

Parameters:
in Input patterns for the currently used model.
target Target values corresponding to the input patterns.
fret Initial error of the currently used model.
ls Type of LineSearch algorithm, that is used by the BFGS algorithm. Possible values: "Dlinmin" (default value), "Linmin", "Cblnsrch"
ax Initial left bracket for the line search algorithms, the default bracket is "0".
bx Initial right bracket for the line search algorithms, the default bracket is "1".
lambda The initial value for the searched $\lambda$, that will minimize the function used by the line search algorithm. The default value is "0.25".
Returns:
none
Author:
C. Igel
Date:
1999
Changes
none
Status
stable
Examples:
KernelOptimization.cpp, LinearClassifierTest.cpp, LinearRegressionTest.cpp, McSvm.cpp, PCAtest.cpp, simpleFFNet.cpp, simpleFFNetSource.cpp, simpleMSERNNet.cpp, simpleRBFNet.cpp, and simpleRNNet.cpp.

Definition at line 321 of file BFGS.cpp.

Referenced by FFNet::activate(), SvmApproximation::addVecFixPointIteration(), FFNet::backprop(), BFGS::bfgs(), BinaryCriterion::binary(), SpanBound1::bound(), SpanBound1::boundDerivative(), CachedMatrix::CachedMatrix(), MSERNNet::calcGradBPTT(), SvmApproximation::calcOffsetForReducedModel(), SvmApproximation::calcOptimalAlphaOfApproximatedSVM(), KernelNearestNeighbor::classify(), Partitioning::Clear(), KernelKMeans::clusterVectors(), GeneralGaussKernel::computeGamma(), QpSvmDecomp::ComputeInnerProduct(), QpBoxAllInOneDecomp::Continue(), QpBoxDecomp::Continue(), CoverTree::CoverTree(), createConnectionMatrixRNN(), Partitioning::CreateIndexed(), SvmApproximation::createIndexListWithKMeans(), SvmApproximation::createIndexListWithStochasticUniversalSampling(), Partitioning::CreateSameSize(), Partitioning::CreateSameSizeBalanced(), CVModel::CVModel(), Dataset::Dataset(), GaussianProcessEvidence::dEvidenceDTheta(), ROC::EqualErrorRate(), WTA::error(), ValidationError::error(), SvmApproximation::error(), SpanBound1::error(), SpanBound::error(), RadiusMargin::error(), Paraboloid::error(), NoisySvmLikelihood::error(), NegativePolarization::error(), NegativeLogLikelihood::error(), MSERNNet::error(), MSERBFNet::error(), LOO::error(), LinearEquation::error(), NegativeBKTA::error(), NegativeKTA::error(), InverseClassSeparability::error(), GaussianProcessVariance::error(), ZeroOneLoss::error(), BalancedClassificationError::error(), ClassificationError::error(), BinaryCriterion::error(), ValidationError::errorDerivative(), SvmApproximationErrorFunctionGlobal::errorDerivative(), SvmApproximationErrorFunction::errorDerivative(), SquaredError::errorDerivative(), SpanBound1::errorDerivative(), RadiusMargin::errorDerivative(), Paraboloid::errorDerivative(), NoisySvmLikelihood::errorDerivative(), NegativePolarization::errorDerivative(), NegativeLogLikelihood::errorDerivative(), MSERBFNet::errorDerivative(), MSEFFNet::errorDerivative(), MeanSquaredError::errorDerivative(), LMSEFFNet::errorDerivative(), LinearEquation::errorDerivative(), NegativeBKTA::errorDerivative(), NegativeKTA::errorDerivative(), InverseClassSeparability::errorDerivative(), CrossEntropyIndependent::errorDerivative(), CrossEntropy::errorDerivative(), MSERNNet::errorPercentage(), VarianceEstimator::estimateFisherInformation(), VarianceEstimator::estimateInvFisher(), WeightedSumKernel2::eval(), WeightedSumKernel::eval(), NormalizedRBFKernel::eval(), RBFKernel::eval(), PolynomialKernel::eval(), LinearKernel::eval(), GeneralGaussKernel::eval(), DiagGaussKernel::eval(), WeightedSumKernel2::evalDerivative(), WeightedSumKernel::evalDerivative(), NormalizedKernel::evalDerivative(), NormalizedRBFKernel::evalDerivative(), RBFKernel::evalDerivative(), KernelFunction::evalDerivative(), GeneralGaussKernel::evalDerivative(), DiagGaussKernel::evalDerivative(), ExpNorm::ExpNorm(), FFNetSource(), FFNet::generalDerivative(), GeneralGaussKernel::GeneralGaussKernel(), Paraboloid::generateBasis(), SvmApproximation::getClassificationRateOnSVsCorrectlyClassifiedByOrigSVM(), DataFile::GetData(), MultiClassTestProblem::GetData(), TransformedProblem::GetData(), SparseDistribution::GetData(), SphereDistribution1::GetData(), NoisyInterval::GetData(), NoisyChessboard::GetData(), Chessboard::GetData(), QpSvmDecomp::getGradient(), MSERNNet::getHistory(), GeneralGaussKernel::getTransformation(), SvmApproximation::gradientDescent(), SVM::ImportLibsvmModel(), SVM::ImportSvmlightModel(), CMAOptimizer::Ind2Model(), Paraboloid::init(), GridSearch::init(), CMAOptimizer::init(), BFGS::initBfgs(), MSERNNet::initWeights(), FFNet::initWeights(), InputLabelMatrix::InputLabelMatrix(), WeightedSumKernel2::isFeasible(), WeightedSumKernel::isFeasible(), JaakkolaHeuristic::JaakkolaHeuristic(), KernelMatrix::KernelMatrix(), CoverTree::KNN(), LinNorm::LinNorm(), SVM::MakeSparse(), PCA::MeanAndScatter(), MultiClassSVM::model(), SVM::model(), LinNorm::model(), ExpNorm::model(), MSERNNet::model(), LinearClassifier::model(), AffineLinearMap::model(), LinearMap::model(), KernelMeanClassifier::model(), FFNet::model(), ComponentWiseModel::model(), SVM::modelDerivative(), LinNorm::modelDerivative(), ExpNorm::modelDerivative(), FFNet::modelDerivative(), MultiClassSVM::MultiClassSVM(), MultiClassSVM::Normalize(), Dataset::NormalizeComponents(), NormalizedKernel::NormalizedKernel(), CoverTree::OneNN(), SVM_Optimizer::optimize(), StochasticGradientDescent::optimize(), SigmoidFit::optimize(), IRpropMinus::optimize(), IRpropPlus::optimize(), RpropMinus::optimize(), RpropPlus::optimize(), QuickpropOriginal::optimize(), Quickprop::optimize(), Perceptron::optimize(), PCA::optimize(), LinearRegression::optimize(), LDA::optimize(), PointSearch::optimize(), NestedGridSearch::optimize(), GridSearch::optimize(), CMAOptimizer::optimize(), AdpBP90b::optimize(), AdpBP90a::optimize(), VarianceEstimator::overallVariance(), Paraboloid::Paraboloid(), MSERNNet::prepareTime(), MSERNNet::processTimeSeries(), MSERNNet::processTimeStep(), QpBoxDecomp::QpBoxDecomp(), QpCrammerSingerDecomp::QpCrammerSingerDecomp(), QPMatrix2::QPMatrix2(), QpSvmDecomp::QpSvmDecomp(), RBFNet::read(), DataFile::ReadExample(), MSERNNet::readParameters(), FFNet::readParameters(), Dataset::ReadSplitFile(), SVM::ReadToken(), DataFile::ReadToken(), KernelNearestNeighbor::Recalc(), KernelMeanClassifier::Recalc(), BFGS::reset(), FFNet::resize(), Dataset::Save(), Dataset::SaveLIBSVM(), CoverTree::SecondNN(), MSERNNet::setHistory(), WeightedSumKernel2::setParameter(), WeightedSumKernel::setParameter(), CVModel::setParameter(), setStructur(), MSERNNet::setStructure(), FFNet::setStructure(), QpCrammerSingerDecomp::Shrink(), Dataset::ShuffleAll(), Dataset::ShuffleTest(), Dataset::ShuffleTraining(), JaakkolaHeuristic::sigma(), QpBoxDecomp::SMO(), QpBoxAndEqDecomp::Solve(), QpBoxAndEqCG::Solve(), QpCrammerSingerDecomp::Solve(), QpBoxAllInOneDecomp::Solve(), QpBoxDecomp::Solve(), QpSvmDecomp::Solve(), QpSvmCG::Solve(), WTA::takeAll(), ROC::Threshold(), GaussianProcess::train(), QpCrammerSingerDecomp::Unshrink(), QpBoxAllInOneDecomp::Unshrink(), QpBoxDecomp::Unshrink(), QpSvmDecomp::Unshrink(), KalmanFilter::updateEstimate(), KernelKMeans::updateFandG(), ROC::Value(), Variance::variance(), MultiClassSVM::VectorToClass(), WeightedSumKernel::WeightedSumKernel(), WeightedSumKernel2::WeightedSumKernel2(), MSERNNet::write(), MSERNNet::writeGradient(), MSERNNet::writeParameters(), FFNet::writeParameters(), InputLabelMatrix::~InputLabelMatrix(), and KernelMatrix::~KernelMatrix().