Click or drag to resize
Accord.NET (logo)

HiddenGradientDescentLearningT Class

Stochastic Gradient Descent learning algorithm for Hidden Conditional Hidden Fields.
Inheritance Hierarchy
SystemObject
  Accord.Statistics.Models.Fields.LearningBaseHiddenConditionalRandomFieldLearningT
    Accord.Statistics.Models.Fields.LearningHiddenGradientDescentLearningT

Namespace:  Accord.Statistics.Models.Fields.Learning
Assembly:  Accord.Statistics (in Accord.Statistics.dll) Version: 3.8.0
Syntax
public class HiddenGradientDescentLearning<T> : BaseHiddenConditionalRandomFieldLearning<T>, 
	ISupervisedLearning<HiddenConditionalRandomField<T>, T[], int>, IParallel, ISupportsCancellation, 
	IHiddenConditionalRandomFieldLearning<T>, IConvergenceLearning, IDisposable
Request Example View Source

Type Parameters

T

The HiddenGradientDescentLearningT type exposes the following members.

Constructors
  NameDescription
Public methodHiddenGradientDescentLearningT
Initializes a new instance of the HiddenGradientDescentLearningT class.
Public methodHiddenGradientDescentLearningT(HiddenConditionalRandomFieldT)
Initializes a new instance of the HiddenGradientDescentLearningT class.
Top
Properties
  NameDescription
Public propertyCurrentIteration
Gets or sets the number of performed iterations.
Public propertyFunction (Inherited from BaseHiddenConditionalRandomFieldLearningT.)
Public propertyHasConverged
Gets or sets whether the algorithm has converged.
Public propertyIterations Obsolete.
Please use MaxIterations instead.
Public propertyLearningRate
Gets or sets the learning rate to use as the gradient descent step size. Default value is 1e-1.
Public propertyMaxIterations
Gets or sets the maximum number of iterations performed by the learning algorithm.
Public propertyModel
Gets or sets the model being trained.
(Inherited from BaseHiddenConditionalRandomFieldLearningT.)
Public propertyParallelOptions
Gets or sets the parallelization options for this algorithm.
Public propertyRegularization
Gets or sets the amount of the parameter weights which should be included in the objective function. Default is 0 (do not include regularization).
Public propertyStochastic
Gets or sets a value indicating whether this HiddenGradientDescentLearningT should use stochastic gradient updates.
Public propertyToken
Gets or sets a cancellation token that can be used to stop the learning algorithm while it is running.
(Inherited from BaseHiddenConditionalRandomFieldLearningT.)
Public propertyTolerance
Gets or sets the maximum change in the average log-likelihood after an iteration of the algorithm used to detect convergence.
Top
Methods
  NameDescription
Protected methodCreate
Creates an instance of the model to be learned. Inheritors of this abstract class must define this method so new models can be created from the training data.
(Inherited from BaseHiddenConditionalRandomFieldLearningT.)
Public methodDispose
Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
Protected methodDispose(Boolean)
Releases unmanaged and - optionally - managed resources
Public methodEquals
Determines whether the specified object is equal to the current object.
(Inherited from Object.)
Protected methodFinalize (Overrides ObjectFinalize.)
Public methodGetHashCode
Serves as the default hash function.
(Inherited from Object.)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Protected methodInnerRun
Runs the learning algorithm.
(Overrides BaseHiddenConditionalRandomFieldLearningTInnerRun(T, Int32).)
Public methodLearn
Learns a model that can map the given inputs to the given outputs.
(Inherited from BaseHiddenConditionalRandomFieldLearningT.)
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Protected methodOnProgressChanged
Raises the [E:ProgressChanged] event.
Public methodReset
Resets the step size.
Public methodRun(T, Int32)
Runs one iteration of the learning algorithm with the specified input training observation and corresponding output label.
Public methodRun(T, Int32) Obsolete.
Runs the learning algorithm with the specified input training observations and corresponding output labels.
Public methodRunEpoch
Runs the learning algorithm with the specified input training observations and corresponding output labels.
Public methodToString
Returns a string that represents the current object.
(Inherited from Object.)
Top
Events
  NameDescription
Public eventProgressChanged
Occurs when the current learning progress has changed.
Top
Extension Methods
  NameDescription
Public Extension MethodHasMethod
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)
Public Extension MethodIsEqual
Compares two objects for equality, performing an elementwise comparison if the elements are vectors or matrices.
(Defined by Matrix.)
Public Extension MethodTo(Type)Overloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Top
Examples
// Let's say we would like to do a very simple mechanism for gesture recognition. 
// In this example, we will be trying to create a classifier that can distinguish 
// between the words "hello", "car", and "wardrobe". 

// Let's say we decided to acquire some data, and we asked some people to perform 
// those words in front of a Kinect camera, and, using Microsoft's SDK, we were able 
// to captured the x and y coordinates of each hand while the word was being performed.

// Let's say we decided to represent our frames as:
// 
//    double[] frame = { leftHandX, leftHandY, rightHandX, rightHandY }; // 4 dimensions
// 
// Since we captured words, this means we captured sequences of frames as we described 
// above. Let's write some of those as rough examples to explain how gesture recognition 
// can be done:

double[][] hello =
{
    new double[] { 1.0, 0.1, 0.0, 0.0 }, // let's say the word
    new double[] { 0.0, 1.0, 0.1, 0.1 }, // hello took 6 frames
    new double[] { 0.0, 1.0, 0.1, 0.1 }, // to be recorded.
    new double[] { 0.0, 0.0, 1.0, 0.0 },
    new double[] { 0.0, 0.0, 1.0, 0.0 },
    new double[] { 0.0, 0.0, 0.1, 1.1 },
};

double[][] car =
{
    new double[] { 0.0, 0.0, 0.0, 1.0 }, // the car word
    new double[] { 0.1, 0.0, 1.0, 0.1 }, // took only 4.
    new double[] { 0.0, 0.0, 0.1, 0.0 },
    new double[] { 1.0, 0.0, 0.0, 0.0 },
};

double[][] wardrobe =
{
    new double[] { 0.0, 0.0, 1.0, 0.0 }, // same for the
    new double[] { 0.1, 0.0, 1.0, 0.1 }, // wardrobe word.
    new double[] { 0.0, 0.1, 1.0, 0.0 },
    new double[] { 0.1, 0.0, 1.0, 0.1 },
};

// Please note that a real-world example would involve *lots* of samples for each word. 
// Here, we are considering just one from each class which is clearly sub-optimal and 
// should _never_ be done on practice. Please keep in mind that we are doing like this
// only to simplify this example on how to create and use HCRFs.

// These are the words we have in our vocabulary:
double[][][] words = { hello, car, wardrobe };

// Now, let's associate integer labels with them. This is needed
// for the case where there are multiple samples for each word.
int[] labels = { 0, 1, 2 };

// Create a new learning algorithm to train the hidden Markov model sequence classifier
var teacher = new HiddenMarkovClassifierLearning<Independent<NormalDistribution>, double[]>()
{
    // Train each model until the log-likelihood changes less than 0.001
    Learner = (i) => new BaumWelchLearning<Independent<NormalDistribution>, double[]>()
    {
        Topology = new Forward(5), // this value can be found by trial-and-error

        // We will create our classifiers assuming an independent Gaussian distribution 
        // for each component in our feature vectors (assuming a Naive Bayes assumption).
        Emissions = (s) => new Independent<NormalDistribution>(dimensions: 4), // 4 dimensions

        Tolerance = 0.001,
        Iterations = 100,

        // This is necessary so the code doesn't blow up when it realizes there is only one 
        // sample per word class. But this could also be needed in normal situations as well:
        FittingOptions = new IndependentOptions()
        {
            InnerOption = new NormalOptions() { Regularization = 1e-5 }
        }
    }
};

// PS: In case you find exceptions trying to configure your model, you might want 
//     to try disabling parallel processing to get more descriptive error messages:
// teacher.ParallelOptions.MaxDegreeOfParallelism = 1;

// Finally, we can run the learning algorithm!
var hmm = teacher.Learn(words, labels);
double logLikelihood = teacher.LogLikelihood;

// At this point, the classifier should be successfully 
// able to distinguish between our three word classes:
// 
int tc1 = hmm.Decide(hello);    // should be 0
int tc2 = hmm.Decide(car);      // should be 1
int tc3 = hmm.Decide(wardrobe); // should be 2
// Now, we can use the Markov classifier to initialize a HCRF
var baseline = HiddenConditionalRandomField.FromHiddenMarkov(hmm);

// We can check that both are equivalent, although they have
// formulations that can be learned with different methods:
int[] predictedLabels = baseline.Decide(words);
// Now we can learn the HCRF using one of the best learning
// algorithms available, Resilient Backpropagation learning:

// Create the Resilient Backpropagation learning algorithm
var rprop = new HiddenResilientGradientLearning<double[]>()
{
    Function = baseline.Function, // use the same HMM function

    Iterations = 50,
    Tolerance = 1e-5
};

// Run the algorithm and learn the models
var hcrf = rprop.Learn(words, labels);

// At this point, the HCRF should be successfully 
// able to distinguish between our three word classes:
// 
int hc1 = hcrf.Decide(hello);    // should be 0
int hc2 = hcrf.Decide(car);      // should be 1
int hc3 = hcrf.Decide(wardrobe); // should be 2
See Also