HiddenMarkovClassifierLearningTDistribution, TObservation Class 
Namespace: Accord.Statistics.Models.Markov.Learning
public class HiddenMarkovClassifierLearning<TDistribution, TObservation> : BaseHiddenMarkovClassifierLearning<HiddenMarkovClassifier<TDistribution, TObservation>, HiddenMarkovModel<TDistribution, TObservation>, TDistribution, TObservation> where TDistribution : Object, IDistribution<TObservation>
The HiddenMarkovClassifierLearningTDistribution, TObservation type exposes the following members.
Name  Description  

HiddenMarkovClassifierLearningTDistribution, TObservation 
Creates a new instance of the learning algorithm for a given
Markov sequence classifier.
 
HiddenMarkovClassifierLearningTDistribution, TObservation(HiddenMarkovClassifierTDistribution, TObservation) 
Creates a new instance of the learning algorithm for a given
Markov sequence classifier.
 
HiddenMarkovClassifierLearningTDistribution, TObservation(HiddenMarkovClassifierTDistribution, TObservation, ClassifierLearningAlgorithmConfiguration)  Obsolete.
Creates a new instance of the learning algorithm for a given
Markov sequence classifier using the specified configuration
function.

Name  Description  

Algorithm  Obsolete.
Obsolete.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Classifier 
Gets the classifier being trained by this instance.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Empirical 
Gets or sets a value indicating whether the class priors
should be estimated from the data, as in an empirical Bayes method.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Learner 
Gets or sets the configuration function specifying which
training algorithm should be used for each of the models
in the hidden Markov model set.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
LogLikelihood 
Gets the loglikelihood at the end of the training.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
ParallelOptions 
Gets or sets the parallelization options for this algorithm.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Rejection 
Gets or sets a value indicating whether a threshold model
should be created or updated after training to support rejection.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Token 
Gets or sets a cancellation token that can be used to
stop the learning algorithm while it is running.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.) 
Name  Description  

Create 
Creates an instance of the model to be learned. Inheritors of this abstract
class must define this method so new models can be created from the training data.
(Overrides BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservationCreate(TObservation, Int32, Int32).)  
CreateThresholdTopology 
Creates the state transition topology for the threshold model. This
method can be used to help in the implementation of the Threshold
abstract method which has to be defined for implementers of this class.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Equals  Determines whether the specified object is equal to the current object. (Inherited from Object.)  
Finalize  Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (Inherited from Object.)  
GetHashCode  Serves as the default hash function. (Inherited from Object.)  
GetType  Gets the Type of the current instance. (Inherited from Object.)  
Learn 
Learns a model that can map the given inputs to the given outputs.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
MemberwiseClone  Creates a shallow copy of the current Object. (Inherited from Object.)  
OnGenerativeClassModelLearningFinished 
Raises the [E:GenerativeClassModelLearningFinished] event.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
OnGenerativeClassModelLearningStarted 
Raises the [E:GenerativeClassModelLearningStarted] event.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
RunT  Obsolete.
Trains each model to recognize each of the output labels.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
Threshold 
Creates a new threshold model
for the current set of Markov models in this sequence classifier.
(Overrides BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservationThreshold.)  
ToString  Returns a string that represents the current object. (Inherited from Object.) 
Name  Description  

ClassModelLearningFinished 
Occurs when the learning of a class model has finished.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.)  
ClassModelLearningStarted 
Occurs when the learning of a class model has started.
(Inherited from BaseHiddenMarkovClassifierLearningTClassifier, TModel, TDistribution, TObservation.) 
Name  Description  

HasMethod 
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)  
IsEqual 
Compares two objects for equality, performing an elementwise
comparison if the elements are vectors or matrices.
(Defined by Matrix.)  
To(Type)  Overloaded.
Converts an object into another type, irrespective of whether
the conversion can be done at compile time or not. This can be
used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)  
ToT  Overloaded.
Converts an object into another type, irrespective of whether
the conversion can be done at compile time or not. This can be
used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.) 
This class acts as a teacher for classifiers based on arbitrarydensity hidden Markov models. The learning algorithm uses a generative approach. It works by training each model in the generative classifier separately.
This can teach models that use any probability distribution. Such arbitrarydensity models can be used for any kind of observation values or vectors. When be used whenever the sequence of observations is discrete or can be represented by discrete symbols, such as class labels, integers, and so on. If you need to classify sequences of other entities, such as real numbers, vectors (i.e. multivariate observations), then you can use genericdensity hidden Markov models. Those models can be modeled after any kind of probability distribution implementing the IDistribution interface.
For a more thorough explanation on hidden Markov models with practical examples on gesture recognition, please see Sequence Classifiers in C#, Part I: Hidden Markov Models [1].
[1]: http://www.codeproject.com/Articles/541428/SequenceClassifiersinCsharpPartIHiddenMarko
The following example creates a continuousdensity hidden Markov model sequence classifier to recognize two classes of univariate observation sequences.
// Create a Continuous density Hidden Markov Model Sequence Classifier // to detect a univariate sequence and the same sequence backwards. double[][] sequences = new double[][] { new double[] { 0,1,2,3,4 }, // This is the first sequence with label = 0 new double[] { 4,3,2,1,0 }, // This is the second sequence with label = 1 }; // Labels for the sequences int[] labels = { 0, 1 }; // Creates a sequence classifier containing 2 hidden Markov Models // with 2 states and an underlying Normal distribution as density. var density = new NormalDistribution(); var classifier = new HiddenMarkovClassifier<NormalDistribution, double>(2, new Ergodic(2), density); // Configure the learning algorithms to train the sequence classifier var teacher = new HiddenMarkovClassifierLearning<NormalDistribution, double>(classifier) { // Train each model until the loglikelihood changes less than 0.001 Learner = modelIndex => new BaumWelchLearning<NormalDistribution, double>(classifier.Models[modelIndex]) { Tolerance = 0.0001, Iterations = 0 } }; // Train the sequence classifier using the algorithm teacher.Learn(sequences, labels); double logLikelihood = teacher.LogLikelihood; // Calculate the probability that the given // sequences originated from the model double likelihood1, likelihood2; // Try to classify the first sequence (output should be 0) int c1 = classifier.Decide(sequences[0]); likelihood1 = classifier.Probability(sequences[0]); // Try to classify the second sequence (output should be 1) int c2 = classifier.Decide(sequences[1]); likelihood2 = classifier.Probability(sequences[1]);
The following example creates a continuousdensity hidden Markov model sequence classifier to recognize two classes of multivariate sequence of observations. This example uses multivariate Normal distributions as emission densities.
When there is insufficient training data, or one of the variables is constant, the Normal distribution estimation may fail with a "Covariance matrix is not positivedefinite". In this case, it is possible to sidestep this issue by specifying a small regularization constant to be added to the diagonal elements of the covariance matrix.
// Create a Continuous density Hidden Markov Model Sequence Classifier // to detect a multivariate sequence and the same sequence backwards. double[][][] sequences = new double[][][] { new double[][] { // This is the first sequence with label = 0 new double[] { 0, 1 }, new double[] { 1, 2 }, new double[] { 2, 3 }, new double[] { 3, 4 }, new double[] { 4, 5 }, }, new double[][] { // This is the second sequence with label = 1 new double[] { 4, 3 }, new double[] { 3, 2 }, new double[] { 2, 1 }, new double[] { 1, 0 }, new double[] { 0, 1 }, } }; // Labels for the sequences int[] labels = { 0, 1 }; // Initial emission density to be copied to each state var initialDensity = new MultivariateNormalDistribution(2); // Creates a sequence classifier containing 2 hidden Markov Models with 2 states // and an underlying multivariate mixture of Normal distributions as density. var classifier = new HiddenMarkovClassifier<MultivariateNormalDistribution, double[]>( classes: 2, topology: new Forward(2), initial: initialDensity); // Configure the learning algorithms to train the sequence classifier var teacher = new HiddenMarkovClassifierLearning<MultivariateNormalDistribution, double[]>(classifier) { // Train each model until the loglikelihood changes less than 0.0001 Learner = modelIndex => new BaumWelchLearning<MultivariateNormalDistribution, double[], NormalOptions>(classifier.Models[modelIndex]) { Tolerance = 0.0001, Iterations = 0, FittingOptions = new NormalOptions() { Diagonal = true, // only diagonal covariance matrices Regularization = 1e5 // avoid nonpositive definite errors } } }; // Train the sequence classifier using the algorithm teacher.Learn(sequences, labels); double logLikelihood = teacher.LogLikelihood; // Calculate the probability that the given // sequences originated from the model double likelihood, likelihood2; int c1 = classifier.Decide(sequences[0]); likelihood = classifier.Probability(sequences[0]); // Try to classify the second sequence (output should be 1) int c2 = classifier.Decide(sequences[1]); likelihood2 = classifier.Probability(sequences[1]);
The next example shows how to use the learning algorithms in a realworld dataset, including training and testing in separate sets and evaluating its performance:
// Ensure we get reproducible results Accord.Math.Random.Generator.Seed = 0; // Download the PENDIGITS dataset from UCI ML repository var pendigits = new Pendigits(path: localDownloadPath); // Get and preprocess the training set double[][][] trainInputs = pendigits.Training.Item1; int[] trainOutputs = pendigits.Training.Item2; // Preprocess the digits so each of them is centered and scaled trainInputs = trainInputs.Apply(Accord.Statistics.Tools.ZScores); // Create some prior distributions to help initialize our parameters var priorC = new WishartDistribution(dimension: 2, degreesOfFreedom: 5); var priorM = new MultivariateNormalDistribution(dimension: 2); // Create a new learning algorithm for creating continuous hidden Markov model classifiers var teacher = new HiddenMarkovClassifierLearning<MultivariateNormalDistribution, double[]>() { // This tells the generative algorithm how to train each of the component models. Note: The learning // algorithm is more efficient if all generic parameters are specified, including the fitting options Learner = (i) => new BaumWelchLearning<MultivariateNormalDistribution, double[], NormalOptions>() { Topology = new Forward(5), // Each model will have a forward topology with 5 states // Their emissions will be multivariate Normal distributions initialized using the prior distributions Emissions = (j) => new MultivariateNormalDistribution(mean: priorM.Generate(), covariance: priorC.Generate()), // We will train until the relative change in the average loglikelihood is less than 1e6 between iterations Tolerance = 1e6, MaxIterations = 1000, // or until we perform 1000 iterations (which is unlikely for this dataset) // We will prevent our covariance matrices from becoming degenerate by adding a small // regularization value to their diagonal until they become positivedefinite again: FittingOptions = new NormalOptions() { Regularization = 1e6 } } }; // The following line is only needed to ensure reproducible results. Please remove it to enable full parallelization teacher.ParallelOptions.MaxDegreeOfParallelism = 1; // (Remove, comment, or change this line to enable full parallelism) // Use the learning algorithm to create a classifier var hmmc = teacher.Learn(trainInputs, trainOutputs); // Compute predictions for the training set int[] trainPredicted = hmmc.Decide(trainInputs); // Check the performance of the classifier by comparing with the groundtruth: var m1 = new GeneralConfusionMatrix(predicted: trainPredicted, expected: trainOutputs); double trainAcc = m1.Accuracy; // should be 0.84962835906232137 // Prepare the testing set double[][][] testInputs = pendigits.Testing.Item1; int[] testOutputs = pendigits.Testing.Item2; // Apply the same normalizations testInputs = testInputs.Apply(Accord.Statistics.Tools.ZScores); // Compute predictions for the test set int[] testPredicted = hmmc.Decide(testInputs); // Check the performance of the classifier by comparing with the groundtruth: var m2 = new GeneralConfusionMatrix(predicted: testPredicted, expected: testOutputs); double testAcc = m2.Accuracy; // should be 0.8130504403522818