1. Regression

Definition

Regression analysis finds the model that best describes how one variable depends on others by drawing a line or curve through the data so it matches the observations as closely as possible. Unlike interpolation, the curve doesn’t have to pass through every point.

Definition – Residual

Given points and a model , the residual at point is

Interpolation (Ch. 3)Regression (Ch. 4)
Passes through all pointsyesno
Number of parameters= number of points< number of points
Good forexact, smooth datanoisy, measured data

2. Choosing a Criterion for the Best Line

We want . What should “best” mean?

Criterion 1: minimize ✗

Example (slides): points .

LineBuilt fromResiduals

Both give the minimum , so the “best” line is not unique. Positive and negative residuals cancel. Bad criterion.

Criterion 2: minimize ✗

Same two lines: for both (and for many other lines). Still not unique. Bad criterion.

Criterion 3: least squares ✓


Squaring punishes large residuals, can’t cancel, and gives a unique line.


3. Least-Squares Straight Line

Derivation

Minimize by setting both partial derivatives to zero:

Theorem – Normal equations

Corollary – Closed-form solution


where and .

⚠️ and are different: square-then-sum vs sum-then-square.

⚠️ The least-squares line always passes through .

Example (extra) – the 4-point data set by least squares

2448
36918
26412
38924
10242662


Line
8
8
(least squares)4 (minimum)

Same ΣEᵢ and Σ|Eᵢ|, different lines: only least squares is unique1.522.533.5246810xyy=4x−4 (Sr=8)y=6 (Sr=8)least squares y=1+2x (Sr=4)


4. Worked Example – Mousetrap Torsion Spring

Example (slides)

Find in .

(rad) (N·m)
0.6981320.1882240.4873880.131405
0.9599310.2091380.9214680.200758
1.1344640.2300521.28700.260986
1.5707960.2509652.46740.394215
1.9198620.3137073.68590.602274
6.28311.19218.84911.5896



Mousetrap spring: T = 0.11767 + 0.096091 θ0.50.91.31.72.10.150.20.250.3θ (rad)T (N·m)fitdata

⚠️ Keep enough digits in the sums: the denominator is a difference of two similar numbers, so rounding the sums too early loses accuracy.