Least Squares (Linear Regression)

$\newcommand{\N}{\mathbb N} \newcommand{\Z}{\mathbb Z} \newcommand{\Q}{\mathbb Q} \newcommand{\R}{\mathbb R} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \definecolor{fillinmathshade}{gray}{0.9} \newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}} $

Section 30.3 Least Squares (Linear Regression)

Suppose we have $N$ data points $(x_i, y_i)$ and want to find the straight line $y = mx + b$ that fits them as closely as possible. The least squares method finds the slope $m$ and intercept $b$ that minimize the sum of the squared vertical distances (shown below in green) between the data points and the line, using these formulas:

Figure 30.1. A least-squares line of best fit through four data points, with vertical distances (residuals) shown in green

$\displaystyle m = \frac{N \sum(x_iy_i) - \sum(x_i) \sum(y_i)}{N \sum(x_i^2) - \left(\sum(x_i)\right)^2}$

$\displaystyle b = \frac{\sum(y_i) - m\sum(x_i)}{N}$

The example below plots a set of data points along with their least-squares line of best fit:

admin.....open in new window

Prev Top Next