Click or drag to resize
Accord.NET (logo) LogisticRegression Class
Binary Logistic Regression.
Inheritance Hierarchy
SystemObject
  Accord.MachineLearningTransformBaseDouble, Boolean
    Accord.MachineLearningClassifierBaseDouble, Boolean
      Accord.MachineLearningBinaryClassifierBaseDouble
        Accord.MachineLearningBinaryScoreClassifierBaseDouble
          Accord.MachineLearningBinaryLikelihoodClassifierBaseDouble
            Accord.Statistics.Models.RegressionGeneralizedLinearRegression
              Accord.Statistics.Models.RegressionLogisticRegression

Namespace:  Accord.Statistics.Models.Regression
Assembly:  Accord.Statistics (in Accord.Statistics.dll) Version: 3.4.0
Syntax
[SerializableAttribute]
public class LogisticRegression : GeneralizedLinearRegression
Request Example View Source

The LogisticRegression type exposes the following members.

Constructors
  NameDescription
Public methodLogisticRegression
Creates a new Logistic Regression Model.
Public methodLogisticRegression(Int32) Obsolete.
Creates a new Logistic Regression Model.
Public methodLogisticRegression(Int32, Double) Obsolete.
Creates a new Logistic Regression Model.
Top
Properties
  NameDescription
Public propertyCoefficients Obsolete.
Obsolete. For quick compatibility fixes in the short term, use GetCoefficient(Int32) and SetCoefficient(Int32, Double).
(Inherited from GeneralizedLinearRegression.)
Public propertyInputs Obsolete.
Gets the number of inputs handled by this model.
(Inherited from GeneralizedLinearRegression.)
Public propertyIntercept
Gets or sets the intercept term. This is always the first value of the Coefficients array.
(Inherited from GeneralizedLinearRegression.)
Public propertyLinear
Gets the underlying linear regression.
(Inherited from GeneralizedLinearRegression.)
Public propertyLink
Gets the link function used by this generalized linear model.
(Inherited from GeneralizedLinearRegression.)
Public propertyNumberOfInputs
Gets the number of inputs accepted by the model.
(Inherited from GeneralizedLinearRegression.)
Public propertyNumberOfOutputs
Gets the number of outputs generated by the model.
(Inherited from TransformBaseTInput, TOutput.)
Public propertyNumberOfParameters
Gets the number of parameters in this model (equals the NumberOfInputs + 1).
(Inherited from GeneralizedLinearRegression.)
Public propertyStandardErrors
Gets the standard errors associated with each coefficient during the model estimation phase.
(Inherited from GeneralizedLinearRegression.)
Public propertyWeights
Gets or sets the linear weights of the regression model. The intercept term is not stored in this vector, but is instead available through the Intercept property.
(Inherited from GeneralizedLinearRegression.)
Top
Methods
  NameDescription
Public methodChiSquare(Double, Double)
The likelihood ratio test of the overall model, also called the model chi-square test.
(Inherited from GeneralizedLinearRegression.)
Public methodChiSquare(Double, Double, Double)
The likelihood ratio test of the overall model, also called the model chi-square test.
(Inherited from GeneralizedLinearRegression.)
Public methodClone
Creates a new GeneralizedLinearRegression that is a copy of the current instance.
(Inherited from GeneralizedLinearRegression.)
Public methodCompute(Double) Obsolete.
Computes the model output for the given input vector.
(Inherited from GeneralizedLinearRegression.)
Public methodCompute(Double) Obsolete.
Computes the model output for each of the given input vectors.
(Inherited from GeneralizedLinearRegression.)
Public methodDecide(TInput)
Computes class-label decisions for a given set of input vectors.
(Inherited from ClassifierBaseTInput, TClasses.)
Public methodDecide(Double)
Computes a class-label decision for a given input.
(Inherited from GeneralizedLinearRegression.)
Public methodDecide(TInput, Boolean)
Computes class-label decisions for the given input.
(Inherited from BinaryClassifierBaseTInput.)
Public methodDecide(TInput, TClasses)
Computes a class-label decision for a given input.
(Inherited from ClassifierBaseTInput, TClasses.)
Public methodEquals
Determines whether the specified object is equal to the current object.
(Inherited from Object.)
Protected methodFinalize
Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection.
(Inherited from Object.)
Public methodStatic memberFromWeights(Double) Obsolete.
Constructs a new LogisticRegression from an array of weights (linear coefficients). The first weight is interpreted as the intercept value.
Public methodStatic memberFromWeights(Double, Double) Obsolete.
Constructs a new LogisticRegression from an array of weights (linear coefficients). The first weight is interpreted as the intercept value.
Public methodGetCoefficient
Gets a coefficient value, where 0 is the intercept term and the other coefficients are indexed starting at 1.
(Inherited from GeneralizedLinearRegression.)
Public methodGetConfidenceInterval(Int32)
Gets the 95% confidence interval for the Odds Ratio for a given coefficient.
Public methodGetConfidenceInterval(Double, Int32, Double, Double)
Gets the confidence interval for an input point.
(Inherited from GeneralizedLinearRegression.)
Public methodGetDegreesOfFreedom
Gets the degrees of freedom when fitting the regression.
(Inherited from GeneralizedLinearRegression.)
Public methodGetDeviance(Double, Double)
Gets the Deviance for the model.
(Inherited from GeneralizedLinearRegression.)
Public methodGetDeviance(Double, Double, Double)
Gets the Deviance for the model.
(Inherited from GeneralizedLinearRegression.)
Public methodGetHashCode
Serves as the default hash function.
(Inherited from Object.)
Public methodGetLogLikelihood(Double, Double)
Gets the Log-Likelihood for the model.
(Inherited from GeneralizedLinearRegression.)
Public methodGetLogLikelihood(Double, Double, Double)
Gets the Log-Likelihood for the model.
(Inherited from GeneralizedLinearRegression.)
Public methodGetLogLikelihoodRatio(Double, Double, GeneralizedLinearRegression)
Gets the Log-Likelihood Ratio between two models.
(Inherited from GeneralizedLinearRegression.)
Public methodGetLogLikelihoodRatio(Double, Double, Double, GeneralizedLinearRegression)
Gets the Log-Likelihood Ratio between two models.
(Inherited from GeneralizedLinearRegression.)
Public methodGetOddsRatio
Gets the Odds Ratio for a given coefficient.
Public methodGetPredictionInterval
Gets the prediction interval for an input point.
(Inherited from GeneralizedLinearRegression.)
Public methodGetPredictionStandardError
Gets the standard error of the prediction for a particular input vector.
(Inherited from GeneralizedLinearRegression.)
Public methodGetStandardError
Gets the standard error of the fit for a particular input vector.
(Inherited from GeneralizedLinearRegression.)
Public methodGetStandardErrors
Gets the standard error for each coefficient.
(Inherited from GeneralizedLinearRegression.)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Public methodGetWaldTest
Gets the Wald Test for a given coefficient.
(Inherited from GeneralizedLinearRegression.)
Public methodLogLikelihood(TInput)
Predicts a class label vector for the given input vector, returning the log-likelihood that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihood(Double)
Predicts a class label vector for the given input vector, returning the log-likelihood that the input vector belongs to its predicted class.
(Inherited from GeneralizedLinearRegression.)
Public methodLogLikelihood(TInput, Boolean)
Predicts a class label for each input vector, returning the log-likelihood that each vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihood(TInput, Double)
Predicts a class label vector for the given input vector, returning the log-likelihood that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihood(Double, Boolean)
Predicts a class label vector for the given input vector, returning the log-likelihood that the input vector belongs to its predicted class.
(Inherited from GeneralizedLinearRegression.)
Public methodLogLikelihood(TInput, Boolean, Double)
Predicts a class label for each input vector, returning the log-likelihood that each vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihoods(TInput, Boolean)
Predicts a class label vector for the given input vector, returning the log-likelihoods of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihoods(TInput, Boolean)
Predicts a class label vector for each input vector, returning the log-likelihoods of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihoods(TInput, Boolean, Double)
Predicts a class label vector for the given input vector, returning the log-likelihoods of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodLogLikelihoods(TInput, Boolean, Double)
Predicts a class label vector for each input vector, returning the log-likelihoods of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Public methodProbabilities(TInput, Boolean)
Predicts a class label vector for the given input vector, returning the probabilities of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbabilities(TInput, Boolean)
Predicts a class label vector for each input vector, returning the probabilities of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbabilities(TInput, Boolean, Double)
Predicts a class label vector for the given input vector, returning the probabilities of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbabilities(TInput, Boolean, Double)
Predicts a class label vector for each input vector, returning the probabilities of the input vector belonging to each possible class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput)
Predicts a class label for the given input vector, returning the probability that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput)
Predicts a class label for the given input vector, returning the probability that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput, Boolean)
Predicts a class label for the given input vector, returning the probability that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput, Boolean)
Predicts a class label for each input vector, returning the probability that each vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput, Double)
Predicts a class label for the given input vector, returning the probability that the input vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodProbability(TInput, Boolean, Double)
Predicts a class label for each input vector, returning the probability that each vector belongs to its predicted class.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodScore(TInput)
Computes a numerical score measuring the association between the given input vector and its most strongly associated class (as predicted by the classifier).
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScore(Double)
Computes a numerical score measuring the association between the given input vector and its most strongly associated class (as predicted by the classifier).
(Inherited from GeneralizedLinearRegression.)
Public methodScore(TInput, Boolean)
Predicts a class label for the input vector, returning a numerical score measuring the strength of association of the input vector to its most strongly related class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScore(TInput, Boolean)
Predicts a class label for each input vector, returning a numerical score measuring the strength of association of the input vector to the most strongly related class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScore(TInput, Double)
Computes a numerical score measuring the association between the given input vector and each class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScore(TInput, Boolean, Double)
Predicts a class label for each input vector, returning a numerical score measuring the strength of association of the input vector to the most strongly related class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput)
Computes a numerical score measuring the association between the given input vector and each class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput)
Computes a numerical score measuring the association between the given input vector and each class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Boolean)
Predicts a class label vector for the given input vector, returning a numerical score measuring the strength of association of the input vector to each of the possible classes.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Double)
Computes a numerical score measuring the association between the given input vector and each class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Boolean)
Predicts a class label vector for each input vector, returning a numerical score measuring the strength of association of the input vector to each of the possible classes.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Double)
Computes a numerical score measuring the association between the given input vector and each class.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Boolean, Double)
Predicts a class label vector for the given input vector, returning a numerical score measuring the strength of association of the input vector to each of the possible classes.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodScores(TInput, Boolean, Double)
Predicts a class label vector for each input vector, returning a numerical score measuring the strength of association of the input vector to each of the possible classes.
(Inherited from BinaryScoreClassifierBaseTInput.)
Public methodSetCoefficient
Sets a coefficient value, where 0 is the intercept term and the other coefficients are indexed starting at 1.
(Inherited from GeneralizedLinearRegression.)
Public methodToMulticlass
Views this instance as a multi-class generative classifier, giving access to more advanced methods, such as the prediction of integer labels.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodToMultilabel
Views this instance as a multi-label generative classifier, giving access to more advanced methods, such as the prediction of one-hot vectors.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodToString
Returns a string that represents the current object.
(Inherited from Object.)
Public methodTransform(TInput)
Applies the transformation to an input, producing an associated output.
(Inherited from ClassifierBaseTInput, TClasses.)
Public methodTransform(TInput)
Applies the transformation to a set of input vectors, producing an associated set of output vectors.
(Inherited from TransformBaseTInput, TOutput.)
Public methodTransform(TInput, Boolean)
Applies the transformation to an input, producing an associated output.
(Inherited from BinaryClassifierBaseTInput.)
Public methodTransform(TInput, Int32)
Applies the transformation to an input, producing an associated output.
(Inherited from BinaryClassifierBaseTInput.)
Public methodTransform(TInput, Boolean)
Applies the transformation to an input, producing an associated output.
(Inherited from BinaryClassifierBaseTInput.)
Public methodTransform(TInput, Int32)
Applies the transformation to an input, producing an associated output.
(Inherited from BinaryClassifierBaseTInput.)
Public methodTransform(TInput, Int32)
Applies the transformation to an input, producing an associated output.
(Inherited from BinaryClassifierBaseTInput.)
Public methodTransform(TInput, Double)
Applies the transformation to a set of input vectors, producing an associated set of output vectors.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodTransform(TInput, Double)
Applies the transformation to a set of input vectors, producing an associated set of output vectors.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodTransform(TInput, Double)
Applies the transformation to a set of input vectors, producing an associated set of output vectors.
(Inherited from BinaryLikelihoodClassifierBaseTInput.)
Public methodTransform(TInput, TClasses)
Applies the transformation to an input, producing an associated output.
(Inherited from ClassifierBaseTInput, TClasses.)
Top
Extension Methods
  NameDescription
Public Extension MethodHasMethod
Checks whether an object implements a method with the given name.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by ExtensionMethods.)
Public Extension MethodToTOverloaded.
Converts an object into another type, irrespective of whether the conversion can be done at compile time or not. This can be used to convert generic types to numeric types during runtime.
(Defined by Matrix.)
Top
Remarks

In statistics, logistic regression (sometimes called the logistic model or Logit model) is used for prediction of the probability of occurrence of an event by fitting data to a logistic curve. It is a generalized linear model used for binomial regression.

Like many forms of regression analysis, it makes use of several predictor variables that may be either numerical or categorical. For example, the probability that a person has a heart attack within a specified time period might be predicted from knowledge of the person's age, sex and body mass index.

Logistic regression is used extensively in the medical and social sciences as well as marketing applications such as prediction of a customer's propensity to purchase a product or cease a subscription.

References:

  • Bishop, Christopher M.; Pattern Recognition and Machine Learning. Springer; 1st ed. 2006.
  • Amos Storkey. (2005). Learning from Data: Learning Logistic Regressors. School of Informatics. Available on: http://www.inf.ed.ac.uk/teaching/courses/lfd/lectures/logisticlearn-print.pdf
  • Cosma Shalizi. (2009). Logistic Regression and Newton's Method. Available on: http://www.stat.cmu.edu/~cshalizi/350/lectures/26/lecture-26.pdf
  • Edward F. Conor. Logistic Regression. Website. Available on: http://userwww.sfsu.edu/~efc/classes/biol710/logistic/logisticreg.htm

Examples

The following example shows how to learn a logistic regression using the standard IterativeReweightedLeastSquares algorithm.

// Suppose we have the following data about some patients.
// The first variable is continuous and represent patient
// age. The second variable is dichotomic and give whether
// they smoke or not (This is completely fictional data).

// We also know if they have had lung cancer or not, and 
// we would like to know whether smoking has any connection
// with lung cancer (This is completely fictional data).

double[][] input =
{              // age, smokes?, had cancer?
    new double[] { 55,    0  }, // false - no cancer
    new double[] { 28,    0  }, // false
    new double[] { 65,    1  }, // false
    new double[] { 46,    0  }, // true  - had cancer
    new double[] { 86,    1  }, // true
    new double[] { 56,    1  }, // true
    new double[] { 85,    0  }, // false
    new double[] { 33,    0  }, // false
    new double[] { 21,    1  }, // false
    new double[] { 42,    1  }, // true
};

bool[] output = // Whether each patient had lung cancer or not
{
    false, false, false, true, true, true, false, false, false, true
};


// To verify this hypothesis, we are going to create a logistic
// regression model for those two inputs (age and smoking), learned
// using a method called "Iteratively Reweighted Least Squares":

var learner = new IterativeReweightedLeastSquares<LogisticRegression>()
{
    Tolerance = 1e-4,  // Let's set some convergence parameters
    Iterations = 100,  // maximum number of iterations to perform
    Regularization = 0
};

// Now, we can use the learner to finally estimate our model:
LogisticRegression regression = learner.Learn(input, output);

// At this point, we can compute the odds ratio of our variables.
// In the model, the variable at 0 is always the intercept term, 
// with the other following in the sequence. Index 1 is the age
// and index 2 is whether the patient smokes or not.

// For the age variable, we have that individuals with
//   higher age have 1.021 greater odds of getting lung
//   cancer controlling for cigarette smoking.
double ageOdds = regression.GetOddsRatio(1); // 1.0208597028836701

// For the smoking/non smoking category variable, however, we
//   have that individuals who smoke have 5.858 greater odds
//   of developing lung cancer compared to those who do not 
//   smoke, controlling for age (remember, this is completely
//   fictional and for demonstration purposes only).
double smokeOdds = regression.GetOddsRatio(2); // 5.8584748789881331

// If we would like to use the model to predict a probability for
// each patient regarding whether they are at risk of cancer or not,
// we can use the Probability function:

double[] scores = regression.Probability(input);

// Finally, if we would like to arrive at a conclusion regarding
// each patient, we can use the Decide method, which will transform
// the probabilities (from 0 to 1) into actual true/false values:

bool[] actual = regression.Decide(input);

Please note that it is also possible to train logistic regression models using large-margin algorithms. With those algorithms, it is possible to train using different regularization options, such as L1 (with ProbabilisticCoordinateDescent) or L2 (with ProbabilisticDualCoordinateDescent). The following example shows how to obtain L1-regularized regression from a probabilistic linear Support Vector Machine:

// Declare some training data. This is exactly the same
// data used in the LogisticRegression documentation page

// Suppose we have the following data about some patients.
// The first variable is continuous and represent patient
// age. The second variable is dichotomic and give whether
// they smoke or not (This is completely fictional data).

// We also know if they have had lung cancer or not, and 
// we would like to know whether smoking has any connection
// with lung cancer (This is completely fictional data).

double[][] input =
{              // age, smokes?, had cancer?
    new double[] { 55,    0  }, // false - no cancer
    new double[] { 28,    0  }, // false
    new double[] { 65,    1  }, // false
    new double[] { 46,    0  }, // true  - had cancer
    new double[] { 86,    1  }, // true
    new double[] { 56,    1  }, // true
    new double[] { 85,    0  }, // false
    new double[] { 33,    0  }, // false
    new double[] { 21,    1  }, // false
    new double[] { 42,    1  }, // true
};

double[] output = // Whether each patient had lung cancer or not
{
    0, 0, 0, 1, 1, 1, 0, 0, 0, 1
};

// Create the L1-regularization learning algorithm
var teacher = new ProbabilisticCoordinateDescent()
{
    Tolerance = 1e-10,
    Complexity = 1e+10, // learn a hard-margin model
};

// Learn the L1-regularized machine
var svm = teacher.Learn(input, output);

// Convert the svm to logistic regression
var regression = (LogisticRegression)svm;

// Compute the predicted outcome for inputs
bool[] predicted = regression.Decide(input);

// Compute log-likelihood scores for the outputs
double[] scores = regression.LogLikelihood(input);

// Compute odds-ratio as in the LogisticRegression example
double ageOdds = regression.GetOddsRatio(1);   // 1.0208597029158772
double smokeOdds = regression.GetOddsRatio(2); // 5.8584748789881331

// Compute the classification error as in SVM example
double error = new ZeroOneLoss(output).Loss(predicted);
See Also