What is the pseudocode for logistic regression

The general process of logistic regression
(1) Data Collection: Collect data using any method.
(2) Prepare data: Since the distance calculation is required, the data type must be numeric. In addition, the structured data format is the best.
(3) Analyze the data: Use any method to analyze the data.
(4) Training algorithm: most of the time is used for training. The purpose of training is to find the best classification regression coefficient.
(5) Test the algorithm: once the training step is completed, the classification is very quick.
(6) Use the algorithm: first we need to enter some data and convert it into appropriate structured values. Then, based on the trained regression coefficients, these values ​​can simply be regression calculation, determine which category they belong to; after that we can do some other analysis work on the output category.
Classification based on logistic regression and sigmoid function
Logistic regression

  • Pros: The calculation costs are not high and are easy to understand and implement.

  • Disadvantages: easy to under-adjust, classification accuracy may not be high.

  • Applicable data types: numeric and nominal data.

The specific calculation formula of the sigmoid function is as follows:

When x is 0, the value of the sigmoid function is 0.5. As x increases, the corresponding sigmoid value approaches 1, and as x decreases, the sigmoid value approaches 0. If the abscissa scale is large enough (Figure 5-1 below), the sigmoid function is similar to a step function

Incline ascent
The gradient ascent algorithm re-estimates the direction of movement after each point has been reached. Starting from P0, after calculating the gradient of this point, the function moves according to the gradient to the next point P1. At point P1 the gradient is recalculated and moves in the new gradient direction to P2. Iterate in this way until the stop condition is met. During the iterative process, the gradient operator always ensures that we can choose the best direction of motion
Gradient Descent Algorithm
The gradient descent algorithm you should hear the most is the same as the gradient ascent algorithm, except that the addition in the formula must be a subtraction. Therefore, the corresponding formula can be written as follows:

The gradient ascent algorithm is used to find the maximum value of the function and the gradient descent algorithm is used to find the minimum value of the function
Training algorithm: use gradient ascent to find the best parameters
The pseudocode of the gradient ascent method is as follows

Training algorithm: stochastic gradient increase
The stochastic gradient ascent algorithm can be written as the following pseudocode:

Example: Predicting the mortality of sick horses from hernias
(1) Data acquisition: given data file.
(2) Prepare the data: use Python to parse the text file and fill in the missing values.
(3) Analyze the data: visualize and observe the data.
(4) Training algorithm: use the optimization algorithm to find the best coefficient.
(5) Test algorithm: in order to quantify the effect of the regression, you need to observe the error rate. Decide whether to resort to the training phase according to the error rate and get better regression coefficients by changing the parameters like number of iterations and step size.
(6) Using the Algorithm: It is not difficult to implement a simple command line program to capture horse symptoms and output prediction results
Prepare the data: Handle missing values ​​in the data

  • Use the mean of the available functions to enter missing values.
  • Use special values ​​to enter missing values, such as: B. 1;
  • Ignore samples with missing values;
  • Use the mean of similar samples to add missing values.
  • Use additional machine learning algorithms to predict missing values.

Test algorithm: use logistic regression for classification
Classifying with the logistic regression method does not require much work. All you have to do is multiply each feature vector in the test set by the regression coefficient obtained by the optimization method, and then multiply by the product.The results are summed and entered into the sigmoid function. If the corresponding sigmoid value is greater than 0.5, the category label is predicted to be 1, otherwise it is predicted to be 0.
Chapter summary

  • The purpose of logistic regression is to find a good fitting parameter for a nonlinear function sigmoid that can be used to optimize the solving process
    Algorithm to complete. Among the optimization algorithms, the gradient ascent algorithm is usually used, and the gradient ascent algorithm can be simplified as a random gradient ascent algorithm.
  • The stochastic gradient ascent algorithm has the same effect as the gradient ascent algorithm, but it requires less computational resources. Then there is the stochastic gradient
    Sheng is an online algorithm that can update parameters when new data arrives without having to reread the entire data set for batch processing.