Click or drag to resize
Accord.NET (logo)

BaumWelchLearning Class

Baum-Welch learning algorithm for discrete-density Hidden Markov Models.
Inheritance Hierarchy
SystemObject
  Accord.MachineLearningParallelLearningBase
    Accord.Statistics.Models.Markov.LearningBaseHiddenMarkovModelLearningHiddenMarkovModel, Int32
      Accord.Statistics.Models.Markov.LearningBaseBaumWelchLearningHiddenMarkovModel, GeneralDiscreteDistribution, Int32, GeneralDiscreteOptions
        Accord.Statistics.Models.Markov.LearningBaumWelchLearning

Namespace:  Accord.Statistics.Models.Markov.Learning
Assembly:  Accord.Statistics (in Accord.Statistics.dll) Version: 3.5.0
Syntax
public class BaumWelchLearning : BaseBaumWelchLearning<HiddenMarkovModel, GeneralDiscreteDistribution, int, GeneralDiscreteOptions>
Request Example View Source

The BaumWelchLearning type exposes the following members.

Constructors
  NameDescription
Public methodBaumWelchLearning
Creates a new instance of the Baum-Welch learning algorithm.
Top
Properties
  NameDescription
Public propertyFittingOptions
Gets or sets the distribution fitting options to use when estimating distribution densities during learning.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyIterations
Gets or sets the maximum number of iterations performed by the learning algorithm.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyLogGamma
Gets the Gamma matrix of log probabilities created during the last iteration of the Baum-Welch learning algorithm.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyLogKsi
Gets the Ksi matrix of log probabilities created during the last iteration of the Baum-Welch learning algorithm.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyLogLikelihood
Gets the log-likelihood of the model at the last iteration.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyLogWeights
Gets the sample weights in the last iteration of the Baum-Welch learning algorithm.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyModel
Gets the model being trained.
(Inherited from BaseHiddenMarkovModelLearningTModel, TObservation.)
Public propertyNumberOfStates
Gets or sets the number of states in the models to be learned.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Protected propertyObservations
Gets all observations as a single vector.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyParallelOptions
Gets or sets the parallelization options for this algorithm.
(Inherited from ParallelLearningBase.)
Public propertyToken
Gets or sets a cancellation token that can be used to cancel the algorithm while it is running.
(Inherited from ParallelLearningBase.)
Public propertyTolerance
Gets or sets the maximum change in the average log-likelihood after an iteration of the algorithm used to detect convergence.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public propertyTopology
Gets or sets the state-transition topology to be used. Default is Ergodic.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Top
Methods
  NameDescription
Protected methodComputeForwardBackward
Computes the forward and backward probabilities matrices for a given observation referenced by its index in the input training data.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Protected methodComputeKsi
Computes the ksi matrix of probabilities for a given observation referenced by its index in the input training data.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public methodEquals
Determines whether the specified object is equal to the current object.
(Inherited from Object.)
Protected methodFinalize
Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection.
(Inherited from Object.)
Protected methodFit
Fits one emission distribution. This method can be override in a base class in order to implement special fitting options.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Public methodStatic memberFromMixtureModel(HiddenMarkovModelMixtureNormalDistribution, NormalOptions)
Creates a Baum-Welch with default configurations for hidden Markov models with normal mixture densities.
Public methodStatic memberFromMixtureModel(HiddenMarkovModelMultivariateMixtureMultivariateNormalDistribution, NormalOptions)
Creates a Baum-Welch with default configurations for hidden Markov models with normal mixture densities.
Public methodGetHashCode
Serves as the default hash function.
(Inherited from Object.)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Public methodLearn
Learns a model that can map the given inputs to the desired outputs.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Public methodRun(Int32) Obsolete.
Obsolete.
Public methodRun(Int32) Obsolete.
Obsolete.
Public methodToString
Returns a string that represents the current object.
(Inherited from Object.)
Protected methodUpdateEmissions
Updates the emission probability matrix.
(Inherited from BaseBaumWelchLearningTModel, TDistribution, TObservation, TOptions.)
Top
Extension Methods
  NameDescription
Public Extension MethodHasMethod
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)
Public Extension MethodIsEqual
Compares two objects for equality, performing an elementwise comparison if the elements are vectors or matrices.
(Defined by Matrix.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by Matrix.)
Top
Remarks

The Baum-Welch algorithm is an unsupervised algorithm used to learn a single hidden Markov model object from a set of observation sequences. It works by using a variant of the Expectation-Maximization algorithm to search a set of model parameters (i.e. the matrix of transition probabilities A, the matrix of emission probabilities B, and the initial probability vector π) that would result in a model having a high likelihood of being able to generate a set of training sequences given to this algorithm.

For increased accuracy, this class performs all computations using log-probabilities.

For a more thorough explanation on hidden Markov models with practical examples on gesture recognition, please see Sequence Classifiers in C#, Part I: Hidden Markov Models [1].

[1]: http://www.codeproject.com/Articles/541428/Sequence-Classifiers-in-Csharp-Part-I-Hidden-Marko

Examples
// We will create a Hidden Markov Model to detect 
// whether a given sequence starts with a zero.
int[][] sequences = new int[][] 
{
    new int[] { 0,1,1,1,1,0,1,1,1,1 },
    new int[] { 0,1,1,1,0,1,1,1,1,1 },
    new int[] { 0,1,1,1,1,1,1,1,1,1 },
    new int[] { 0,1,1,1,1,1         },
    new int[] { 0,1,1,1,1,1,1       },
    new int[] { 0,1,1,1,1,1,1,1,1,1 },
    new int[] { 0,1,1,1,1,1,1,1,1,1 },
};

// Create a new Hidden Markov Model with 3 states for
//  an output alphabet of two characters (zero and one)
var hmm = new HiddenMarkovModel(states: 3, symbols: 2);

// Create the learning algorithm
var teacher = new BaumWelchLearning(hmm)
{
    Tolerance = 0.0001, // until log-likelihood changes less than 0.0001
    Iterations = 0      // and use as many iterations as needed
};

// Estimate the model
teacher.Learn(sequences);


// Now we can calculate the probability that the given
// sequences originated from the model. We can compute
// those probabilities using the Viterbi algorithm:
double vl1; hmm.Decode(new int[] { 0, 1 }, out vl1);        // -0.69317855
double vl2; hmm.Decode(new int[] { 0, 1, 1, 1 }, out vl2);  // -2.16644878

// Sequences which do not start with zero have much lesser probability.
double vl3; hmm.Decode(new int[] { 1, 1 }, out vl3);        // -11.3580034
double vl4; hmm.Decode(new int[] { 1, 0, 0, 0 }, out vl4);  // -38.6759130

// Sequences which contains few errors have higher probability
//  than the ones which do not start with zero. This shows some
//  of the temporal elasticity and error tolerance of the HMMs.
double vl5; hmm.Decode(new int[] { 0, 1, 0, 1, 1, 1, 1, 1, 1 }, out vl5); // -8.22665
double vl6; hmm.Decode(new int[] { 0, 1, 1, 1, 1, 1, 1, 0, 1 }, out vl6); // -8.22665


// Additionally, we can also compute the probability 
// of those sequences using the forward algorithm:
double fl1 = hmm.LogLikelihood(new int[] { 0, 1 });        // -0.000031369
double fl2 = hmm.LogLikelihood(new int[] { 0, 1, 1, 1 });  // -0.087005121

// Sequences which do not start with zero have much lesser probability.
double fl3 = hmm.LogLikelihood(new int[] { 1, 1 });        // -10.66485629
double fl4 = hmm.LogLikelihood(new int[] { 1, 0, 0, 0 });  // -36.61788687

// Sequences which contains few errors have higher probability
//  than the ones which do not start with zero. This shows some
//  of the temporal elasticity and error tolerance of the HMMs.
double fl5 = hmm.LogLikelihood(new int[] { 0, 1, 0, 1, 1, 1, 1, 1, 1 }); // -3.3744416
double fl6 = hmm.LogLikelihood(new int[] { 0, 1, 1, 1, 1, 1, 1, 0, 1 }); // -3.3744416
See Also