Machine Learning - Simple linear regression（Principle）

Simple linear regression：

Known data sets $D = {(x_1,y_2),(x_2,y_2)....(x_n,y_n)}$ There is only one self-variable x with a variable y. And x and y satisfy a linear relationship.

For these questions，we are trying to fit the following equation:

Simultaneously make $f(x_i) \doteq y_i$ in the above equation,It can be seen from this that determining w and b is the most important issue among them.

In the regression task, we use mean squared error as a measure of performance, so we need to determine how much the mean squared error can be minimized when w and b are equal to:

$(w^{*},b^{*}) = argmin_{(w,b)}\sum_{i = 1}^{m}(f(x_i) - y_i)^{2}$

$=argmin_{(w,b)}\sum_{i = 1}^{m}(y_i-wx_i - y_i)^{2}$

In the regression task, we use mean square error as a measure of performance, so we need to find the minimum mean square error when w and b are equal. This model based on minimizing mean square error is called the "least squares method", where the formula for mean square error corresponds exactly to the Euclidean distance. The purpose of this algorithm is to try to find a line that minimizes the sum of Euclidean distance lines for the sample points.

The process of solving w and b is actually finding a function

$E(w,b)=argmin_{(w,b)}\sum_{i = 1}^{m}(y_i-wx_i - y_i)^{2}$

The process of minimization is called the least squares Parameter Estimation of minimum linear regression.

Based on the knowledge of advanced mathematics, we can know that for a multivariate function to find its maximum value, it is to take the partial derivative of the parameter and make it 0,then set the partial derivatives to 0 separately, and finally solve for w and b:

$b = \frac{1}{m} \sum_{i = 1}^{m} (y_i - w_i)$

$w = \frac{ \sum_{i = 1}^{m}y_i(x_i - \bar{x}) }{ \sum_{i = 1}^{m}x_i^{2} - \frac{1}{m}(\sum_{i = 1}^{m}x_i)^{2} }$

This is the result we need to seek.

秒客网

Machine Learning - Simple linear regression（Principle）

Simple linear regression：

Known data sets $D = {(x_1,y_2),(x_2,y_2)....(x_n,y_n)}$ There is only one self-variable x with a variable y. And x and y satisfy a linear relationship.

相关文章

Machine Learning - Simple linear regression（Principle）

Simple linear regression：

Known data sets There is only one self-variable x with a variable y. And x and y satisfy a linear relationship.

相关文章

Known data sets $D = {(x_1,y_2),(x_2,y_2)....(x_n,y_n)}$ There is only one self-variable x with a variable y. And x and y satisfy a linear relationship.