Click or drag to resize
Accord.NET (logo)

AdaBoostTModel Class

AdaBoost learning algorithm.
Inheritance Hierarchy
SystemObject
  Accord.MachineLearning.BoostingAdaBoostTModel

Namespace:  Accord.MachineLearning.Boosting
Assembly:  Accord.MachineLearning (in Accord.MachineLearning.dll) Version: 3.8.0
Syntax
public class AdaBoost<TModel> : ISupervisedBinaryLearning<Boost<TModel>, double[]>, 
	ISupervisedMulticlassLearning<Boost<TModel>, double[]>, ISupervisedMultilabelLearning<Boost<TModel>, double[]>, 
	ISupervisedLearning<Boost<TModel>, double[], int[]>, ISupervisedLearning<Boost<TModel>, double[], bool[]>, 
	ISupervisedLearning<Boost<TModel>, double[], int>, ISupervisedLearning<Boost<TModel>, double[], bool>
where TModel : Object, IClassifier<double[], int>
Request Example View Source

Type Parameters

TModel
The type of the model to be trained.

The AdaBoostTModel type exposes the following members.

Constructors
  NameDescription
Public methodAdaBoostTModel
Initializes a new instance of the AdaBoostTModel class.
Public methodAdaBoostTModel(BoostTModel)
Initializes a new instance of the AdaBoostTModel class.
Public methodAdaBoostTModel(BoostTModel, ModelConstructorTModel) Obsolete.
Initializes a new instance of the AdaBoostTModel class.
Top
Properties
  NameDescription
Public propertyCreation Obsolete.
Gets or sets the fitting function which creates and trains a model given a weighted data set.
Public propertyIterations Obsolete.
Please use MaxIterations instead.
Public propertyLearner
Gets or sets a function that takes a set of parameters and creates a learning algorithm for learning each stage of the boosted classsifier.
Public propertyMaxIterations
Gets or sets the maximum number of iterations performed by the learning algorithm.
Public propertyModel
Gets or sets the model being trained.
Public propertyThreshold
Gets or sets the error limit before learning stops. Default is 0.5.
Public propertyToken
Gets or sets a cancellation token that can be used to stop the learning algorithm while it is running.
Public propertyTolerance
Gets or sets the relative tolerance used to detect convergence of the learning algorithm.
Top
Methods
  NameDescription
Public methodComputeError
Computes the error ratio, the number of misclassifications divided by the total number of samples in a dataset.
Public methodEquals
Determines whether the specified object is equal to the current object.
(Inherited from Object.)
Protected methodFinalize
Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection.
(Inherited from Object.)
Public methodGetHashCode
Serves as the default hash function.
(Inherited from Object.)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Public methodLearn(Double, Boolean, Double)
Learns a model that can map the given inputs to the given outputs.
Public methodLearn(Double, Boolean, Double)
Learns a model that can map the given inputs to the given outputs.
Public methodLearn(Double, Int32, Double)
Learns a model that can map the given inputs to the given outputs.
Public methodLearn(Double, Int32, Double)
Learns a model that can map the given inputs to the given outputs.
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Public methodRun(Double, Int32) Obsolete.
Runs the learning algorithm.
Public methodRun(Double, Int32, Double) Obsolete.
Runs the learning algorithm.
Public methodToString
Returns a string that represents the current object.
(Inherited from Object.)
Top
Extension Methods
  NameDescription
Public Extension MethodHasMethod
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)
Public Extension MethodIsEqual
Compares two objects for equality, performing an elementwise comparison if the elements are vectors or matrices.
(Defined by Matrix.)
Public Extension MethodTo(Type)Overloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Top
Remarks

AdaBoost, short for "Adaptive Boosting", is a machine learning meta-algorithm formulated by Yoav Freund and Robert Schapire who won the Gödel Prize in 2003 for their work. It can be used in conjunction with many other types of learning algorithms to improve their performance. The output of the other learning algorithms ('weak learners') is combined into a weighted sum that represents the final output of the boosted classifier. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers. In some problems it can be less susceptible to the overfitting problem than other learning algorithms. The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing (e.g., their error rate is smaller than 0.5 for binary classification), the final model can be proven to converge to a strong learner.

Every learning algorithm will tend to suit some problem types better than others, and will typically have many different parameters and configurations to be adjusted before achieving optimal performance on a dataset, AdaBoost(with decision trees as the weak learners) is often referred to as the best out-of-the-box classifier. When used with decision tree learning, information gathered at each stage of the AdaBoost algorithm about the relative 'hardness' of each training sample is fed into the tree growing algorithm such that later trees tend to focus on harder-to-classify examples.

References:

Examples
// Let's say we want to classify the following 2-dimensional 
// data samples into 2 possible classes, either true or false:
double[][] inputs =
{
    new double[] {  10, 42 },
    new double[] { 162, 96 },
    new double[] { 125, 20 },
    new double[] {  96,  6 },
    new double[] {   2, 73 },
    new double[] {  52, 51 },
    new double[] {  71, 49 },
};

// And those are their associated class labels
bool[] outputs =
{
    false, false, true, true, false, false, true
};

// We can create an AdaBoost algorithm as:
var learner = new AdaBoost<DecisionStump>()
{
    Learner = (p) => new ThresholdLearning(),

    // Train until:
    MaxIterations = 5,
    Tolerance = 1e-3
};

// Now, we can use the Learn method to learn a boosted classifier
Boost<DecisionStump> classifier = learner.Learn(inputs, outputs);

// And we can test its performance using (error should be 0):
ConfusionMatrix cm = ConfusionMatrix.Estimate(classifier, inputs, outputs);

double error = cm.Error;  // should be 0.0
double acc = cm.Accuracy; // should be 1.0
double kappa = cm.Kappa;  // should be 1.0

// And compute a decision for a single data point using:
bool y = classifier.Decide(inputs[0]); // result should false
// This example shows how to use AdaBoost to train more complex
// models than a simple DecisionStump. For example, we will use
// it to train a boosted Logistic Regression classifier.

// Let's use some synthetic data for that: The Yin-Yang dataset is 
// a simple 2D binary non-linear decision problem where the points 
// belong to each of the classes interwine in a Yin-Yang shape:
var dataset = new YinYang();
double[][] inputs = dataset.Instances;
int[] outputs = Classes.ToZeroOne(dataset.ClassLabels);

// Create an AdaBoost for Logistic Regression as:
var teacher = new AdaBoost<LogisticRegression>()
{
    // Here we can specify how each regression should be learned:
    Learner = (param) => new IterativeReweightedLeastSquares<LogisticRegression>()
    {
        ComputeStandardErrors = false,
        MaxIterations = 50,
        Tolerance = 0
    },

    // Train until:
    MaxIterations = 50,
    Tolerance = 1e-5,
};

// Now, we can use the Learn method to learn a boosted classifier
Boost<LogisticRegression> classifier = teacher.Learn(inputs, outputs);

// And we can test its performance using (error should be 0.11):
ConfusionMatrix cm = ConfusionMatrix.Estimate(classifier, inputs, outputs);

double error = cm.Error;  // should be 0.11
double acc = cm.Accuracy; // should be 0.89
double kappa = cm.Kappa;  // should be 0.78

// And compute a decision for a single data point using:
bool y = classifier.Decide(inputs[0]); // result should false
// This example shows how to use AdaBoost to train more complex
// models than a simple DecisionStump. For example, we will use
// it to train a boosted Decision Trees.

// Let's use some synthetic data for that: The Yin-Yang dataset is 
// a simple 2D binary non-linear decision problem where the points 
// belong to each of the classes interwine in a Yin-Yang shape:
var dataset = new YinYang();
double[][] inputs = dataset.Instances;
int[] outputs = Classes.ToZeroOne(dataset.ClassLabels);

// Create an AdaBoost for Logistic Regression as:
var teacher = new AdaBoost<DecisionTree>()
{
    // Here we can specify how each regression should be learned:
    Learner = (param) => new C45Learning()
    {
        // i.e.
        // MaxHeight = 
        // MaxVariables = 
    },

    // Train until:
    MaxIterations = 50,
    Tolerance = 1e-5,
};

// Now, we can use the Learn method to learn a boosted classifier
Boost<DecisionTree> classifier = teacher.Learn(inputs, outputs);

// And we can test its performance using (error should be 0.11):
double error = ConfusionMatrix.Estimate(classifier, inputs, outputs).Error;

// And compute a decision for a single data point using:
bool y = classifier.Decide(inputs[0]); // result should false
See Also