1. Regression
Definition
Regression analysis finds the model that best describes how one variable depends on others by drawing a line or curve through the data so it matches the observations as closely as possible. Unlike interpolation, the curve doesn’t have to pass through every point.
Definition – Residual
Given points and a model , the residual at point is
| Interpolation (Ch. 3) | Regression (Ch. 4) | |
|---|---|---|
| Passes through all points | yes | no |
| Number of parameters | = number of points | < number of points |
| Good for | exact, smooth data | noisy, measured data |
2. Choosing a Criterion for the Best Line
We want . What should “best” mean?
Criterion 1: minimize ✗
Example (slides): points .
| Line | Built from | Residuals | |
|---|---|---|---|
Both give the minimum , so the “best” line is not unique. Positive and negative residuals cancel. Bad criterion.
Criterion 2: minimize ✗
Same two lines: for both (and for many other lines). Still not unique. Bad criterion.
Criterion 3: least squares ✓
Squaring punishes large residuals, can’t cancel, and gives a unique line.
3. Least-Squares Straight Line
Derivation
Minimize by setting both partial derivatives to zero:
Theorem – Normal equations
Corollary – Closed-form solution
where and .
⚠️ and are different: square-then-sum vs sum-then-square.
⚠️ The least-squares line always passes through .
Example (extra) – the 4-point data set by least squares
| 2 | 4 | 4 | 8 |
| 3 | 6 | 9 | 18 |
| 2 | 6 | 4 | 12 |
| 3 | 8 | 9 | 24 |
| 10 | 24 | 26 | 62 |
| Line | |
|---|---|
| 8 | |
| 8 | |
| (least squares) | 4 (minimum) |
4. Worked Example – Mousetrap Torsion Spring
Example (slides)
Find in .
| (rad) | (N·m) | ||
|---|---|---|---|
| 0.698132 | 0.188224 | 0.487388 | 0.131405 |
| 0.959931 | 0.209138 | 0.921468 | 0.200758 |
| 1.134464 | 0.230052 | 1.2870 | 0.260986 |
| 1.570796 | 0.250965 | 2.4674 | 0.394215 |
| 1.919862 | 0.313707 | 3.6859 | 0.602274 |
| 6.2831 | 1.1921 | 8.8491 | 1.5896 |
⚠️ Keep enough digits in the sums: the denominator is a difference of two similar numbers, so rounding the sums too early loses accuracy.