LevenbergMarquardtLearning Class 
Namespace: Accord.Neuro.Learning
The LevenbergMarquardtLearning type exposes the following members.
Name  Description  

LevenbergMarquardtLearning(ActivationNetwork) 
Initializes a new instance of the LevenbergMarquardtLearning class.
 
LevenbergMarquardtLearning(ActivationNetwork, JacobianMethod) 
Initializes a new instance of the LevenbergMarquardtLearning class.
 
LevenbergMarquardtLearning(ActivationNetwork, Boolean) 
Initializes a new instance of the LevenbergMarquardtLearning class.
 
LevenbergMarquardtLearning(ActivationNetwork, Boolean, JacobianMethod) 
Initializes a new instance of the LevenbergMarquardtLearning class.

Name  Description  

Adjustment 
Learning rate adjustment. Default value is 10.
 
Alpha 
Gets or sets the importance of the squared sum of network
weights in the cost function. Used by the regularization.
 
Beta 
Gets or sets the importance of the squared sum of network
errors in the cost function. Used by the regularization.
 
Blocks 
Gets or sets the number of blocks to divide the
Jacobian matrix in the Hessian calculation to
preserve memory. Default is 1.
 
EffectiveParameters 
Gets the number of effective parameters being used
by the network as determined by the Bayesian regularization.
 
Gradient 
Gets the gradient vector computed in the last iteration.
 
Hessian 
Gets the approximate Hessian matrix of second derivatives
generated in the last algorithm iteration. The Hessian is
stored in the upper triangular part of this matrix. See
remarks for details.
 
Jacobian 
Gets the Jacobian matrix created in the last iteration.
 
LearningRate 
Levenberg's damping factor (lambda). This
value must be positive. Default is 0.1.
 
NumberOfParameters 
Gets the total number of parameters
in the network being trained.
 
ParallelOptions 
Gets or sets the parallelization options for this algorithm.
 
UseRegularization 
Gets or sets whether to use Bayesian Regularization.

Name  Description  

ComputeError 
Compute network error for a given data set.
 
Equals  Determines whether the specified object is equal to the current object. (Inherited from Object.)  
Finalize  Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (Inherited from Object.)  
GetHashCode  Serves as the default hash function. (Inherited from Object.)  
GetType  Gets the Type of the current instance. (Inherited from Object.)  
MemberwiseClone  Creates a shallow copy of the current Object. (Inherited from Object.)  
Run 
This method should not be called. Use RunEpoch(Double, Double) instead.
 
RunEpoch 
Runs a single learning epoch.
 
ToString  Returns a string that represents the current object. (Inherited from Object.) 
Name  Description  

HasMethod 
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)  
IsEqual 
Compares two objects for equality, performing an elementwise
comparison if the elements are vectors or matrices.
(Defined by Matrix.)  
To(Type)  Overloaded.
Converts an object into another type, irrespective of whether
the conversion can be done at compile time or not. This can be
used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)  
ToT  Overloaded.
Converts an object into another type, irrespective of whether
the conversion can be done at compile time or not. This can be
used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.) 
This class implements the LevenbergMarquardt learning algorithm, which treats the neural network learning as a function optimization problem. The LevenbergMarquardt is one of the fastest and accurate learning algorithms for small to medium sized networks.
However, in general, the standard LM algorithm does not perform as well on pattern recognition problems as it does on function approximation problems. The LM algorithm is designed for least squares problems that are approximately linear. Because the output neurons in pattern recognition problems are generally saturated, it will not be operating in the linear region.
The advantages of the LM algorithm decreases as the number of network parameters increases.
Sample usage (training network to calculate XOR function):
// initialize input and output values double[][] input = { new double[] {0, 0}, new double[] {0, 1}, new double[] {1, 0}, new double[] {1, 1} }; double[][] output = { new double[] {0}, new double[] {1}, new double[] {1}, new double[] {0} }; // create neural network ActivationNetwork network = new ActivationNetwork( SigmoidFunction( 2 ), 2, // two inputs in the network 2, // two neurons in the first layer 1 ); // one neuron in the second layer // create teacher LevenbergMarquardtLearning teacher = new LevenbergMarquardtLearning( network ); // loop while ( !needToStop ) { // run epoch of learning procedure double error = teacher.RunEpoch( input, output ); // check error value to see if we need to stop // ... }
The following example shows how to create a neural network to learn a classification problem with multiple classes.
// Here we will be creating a neural network to process 3valued input // vectors and classify them into 4possible classes. We will be using // a single hidden layer with 5 hidden neurons to accomplish this task. // int numberOfInputs = 3; int numberOfClasses = 4; int hiddenNeurons = 5; // Those are the input vectors and their expected class labels // that we expect our network to learn. // double[][] input = { new double[] { 1, 1, 1 }, // 0 new double[] { 1, 1, 1 }, // 1 new double[] { 1, 1, 1 }, // 1 new double[] { 1, 1, 1 }, // 0 new double[] { 1, 1, 1 }, // 2 new double[] { 1, 1, 1 }, // 3 new double[] { 1, 1, 1 }, // 3 new double[] { 1, 1, 1 } // 2 }; int[] labels = { 0, 1, 1, 0, 2, 3, 3, 2, }; // In order to perform multiclass classification, we have to select a // decision strategy in order to be able to interpret neural network // outputs as labels. For this, we will be expanding our 4 possible class // labels into 4dimensional output vectors where one single dimension // corresponding to a label will contain the value +1 and 1 otherwise. double[][] outputs = Accord.Statistics.Tools .Expand(labels, numberOfClasses, 1, 1); // Next we can proceed to create our network var function = new BipolarSigmoidFunction(2); var network = new ActivationNetwork(function, numberOfInputs, hiddenNeurons, numberOfClasses); // Heuristically randomize the network new NguyenWidrow(network).Randomize(); // Create the learning algorithm var teacher = new LevenbergMarquardtLearning(network); // Teach the network for 10 iterations: double error = Double.PositiveInfinity; for (int i = 0; i < 10; i++) error = teacher.RunEpoch(input, outputs); // At this point, the network should be able to // perfectly classify the training input points. for (int i = 0; i < input.Length; i++) { int answer; double[] output = network.Compute(input[i]); double response = output.Max(out answer); int expected = labels[i]; // at this point, the variables 'answer' and // 'expected' should contain the same value. }
References: