Click or drag to resize
Accord.NET (logo) CrossValidation Class
k-Fold cross-validation.
Inheritance Hierarchy
SystemObject
  Accord.MachineLearningCrossValidationObject
    Accord.MachineLearningCrossValidation

Namespace:  Accord.MachineLearning
Assembly:  Accord.MachineLearning (in Accord.MachineLearning.dll) Version: 3.4.0
Syntax
[SerializableAttribute]
public class CrossValidation : CrossValidation<Object>
Request Example View Source

The CrossValidation type exposes the following members.

Constructors
Properties
  NameDescription
Public propertyFitting
Gets or sets the model fitting function.
(Inherited from CrossValidationTModel.)
Public propertyFolds
Gets the array of data set indexes contained in each fold.
(Inherited from CrossValidationTModel.)
Public propertyIndices
Gets the array of fold indices for each point in the data set.
(Inherited from CrossValidationTModel.)
Public propertyK
Gets the number of folds in the k-fold cross validation.
(Inherited from CrossValidationTModel.)
Public propertyRunInParallel
Gets or sets a value indicating whether to use parallel processing through the use of multiple threads or not. Default is true.
(Inherited from CrossValidationTModel.)
Public propertySamples
Gets the total number of data samples in the data set.
(Inherited from CrossValidationTModel.)
Top
Methods
  NameDescription
Public methodCompute
Computes the cross validation algorithm.
(Inherited from CrossValidationTModel.)
Public methodCreatePartitions
Gets the indices for the training and validation sets for the specified validation fold index.
(Inherited from CrossValidationTModel.)
Public methodEquals
Determines whether the specified object is equal to the current object.
(Inherited from Object.)
Protected methodFinalize
Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection.
(Inherited from Object.)
Public methodGetHashCode
Serves as the default hash function.
(Inherited from Object.)
Public methodGetPartitionSize
Gets the number of instances in training and validation sets for the specified validation fold index.
(Inherited from CrossValidationTModel.)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Public methodStatic memberSplittings(Int32, Int32)
Create cross-validation folds by generating a vector of random fold indices.
Public methodStatic memberSplittings(Int32, Int32, Int32)
Create cross-validation folds by generating a vector of random fold indices, making sure class labels get equally distributed among the folds.
Public methodToString
Returns a string that represents the current object.
(Inherited from Object.)
Top
Extension Methods
  NameDescription
Public Extension MethodHasMethod
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by Matrix.)
Top
Remarks

Cross-validation is a technique for estimating the performance of a predictive model. It can be used to measure how the results of a statistical analysis will generalize to an independent data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.

One round of cross-validation involves partitioning a sample of data into complementary subsets, performing the analysis on one subset (called the training set), and validating the analysis on the other subset (called the validation set or testing set). To reduce variability, multiple rounds of cross-validation are performed using different partitions, and the validation results are averaged over the rounds.

References:

Examples
// This is a sample code on how to use Cross-Validation
// to assess the performance of Support Vector Machines.

// Consider the example binary data. We will be trying
// to learn a XOR problem and see how well does SVMs
// perform on this data.

double[][] data =
{
    new double[] { -1, -1 }, new double[] {  1, -1 },
    new double[] { -1,  1 }, new double[] {  1,  1 },
    new double[] { -1, -1 }, new double[] {  1, -1 },
    new double[] { -1,  1 }, new double[] {  1,  1 },
    new double[] { -1, -1 }, new double[] {  1, -1 },
    new double[] { -1,  1 }, new double[] {  1,  1 },
    new double[] { -1, -1 }, new double[] {  1, -1 },
    new double[] { -1,  1 }, new double[] {  1,  1 },
};

int[] xor = // result of xor for the sample input data
{
    -1,       1,
     1,      -1,
    -1,       1,
     1,      -1,
    -1,       1,
     1,      -1,
    -1,       1,
     1,      -1,
};


// Create a new Cross-validation algorithm passing the data set size and the number of folds
var crossvalidation = new CrossValidation(size: data.Length, folds: 3);

// Define a fitting function using Support Vector Machines. The objective of this
// function is to learn a SVM in the subset of the data indicated by cross-validation.

crossvalidation.Fitting = delegate(int k, int[] indicesTrain, int[] indicesValidation)
{
    // The fitting function is passing the indices of the original set which
    // should be considered training data and the indices of the original set
    // which should be considered validation data.

    // Lets now grab the training data:
    var trainingInputs = data.Submatrix(indicesTrain);
    var trainingOutputs = xor.Submatrix(indicesTrain);

    // And now the validation data:
    var validationInputs = data.Submatrix(indicesValidation);
    var validationOutputs = xor.Submatrix(indicesValidation);


    // Create a Kernel Support Vector Machine to operate on the set
    var svm = new KernelSupportVectorMachine(new Polynomial(2), 2);

    // Create a training algorithm and learn the training data
    var smo = new SequentialMinimalOptimization(svm, trainingInputs, trainingOutputs);

    double trainingError = smo.Run();

    // Now we can compute the validation error on the validation data:
    double validationError = smo.ComputeError(validationInputs, validationOutputs);

    // Return a new information structure containing the model and the errors achieved.
    return new CrossValidationValues(svm, trainingError, validationError);
};


// Compute the cross-validation
var result = crossvalidation.Compute();

// Finally, access the measured performance.
double trainingErrors = result.Training.Mean;
double validationErrors = result.Validation.Mean;
See Also