1. Statistical Inference
Definition — Statistical Inference
Drawing conclusions about population parameters from sample data. Two main branches:
| Branch | Question | Output | Chapters |
|---|---|---|---|
| Estimation | ”What is the value of ?“ | a point estimate or an interval | 12–13 |
| Hypothesis testing | ”Is believable?“ | accept / reject a claim | 14 |
Classical vs. Bayesian
The classical (frequentist) approach treats parameters as fixed unknown constants and bases inference only on the sample. The Bayesian approach also uses prior beliefs about the parameter. This course uses the classical approach.
2. Methods of Estimation: Point Estimation
Definition — Estimator and Estimate
An estimator is a statistic used to estimate a parameter (a random variable). Its observed value is the point estimate.
Examples: estimates ; estimates ; estimates .
Definition — Unbiased Estimator
is unbiased for if . The bias is .
- is unbiased for ; is unbiased for (proved in Chapter 10).
- is biased: .
Definition — Efficiency and MVUE
Among unbiased estimators, the one with the smallest variance is the most efficient — the minimum variance unbiased estimator (MVUE).
For a normal population, both and the sample median are unbiased for , but , so is more efficient.
Definition — Maximum Likelihood Estimation
Given data , the likelihood is . The maximum likelihood estimator is the value that maximizes (usually by setting ).
Example. Exponential lifetimes with mean : . Setting gives .
3. Methods of Estimation: Interval Estimation
Definition — Confidence Interval
An interval computed from the sample such that, before sampling,
is a confidence interval; is the confidence coefficient.
Interpretation
If we repeated the sampling many times and built an interval each time, about of the intervals would contain the true .
⚠️ Once computed, a specific interval like either contains or not. It is wrong to say “there is a 95% probability that is in ”; the 95% describes the method.
Trade-offs
- Higher confidence → wider interval.
- Larger → narrower interval (width ).
4. Estimating the Mean — Known
Theorem — CI for , Known
If is the mean of a random sample of size from a population with known (normal population, or large):
Derivation. , so
Multiply through by , subtract and multiply by (which flips the inequalities) to isolate .
| Confidence | 90% | 95% | 99% |
|---|---|---|---|
| 1.645 | 1.960 | 2.576 |
Worked Example
Zinc concentration from locations: g/mL; known. .
- 95%:
- 99%:
Theorem — Error Bound and Sample Size
With confidence, the error does not exceed . To guarantee a maximum error :
Example. For error at most 0.05 with 95% confidence: .
⚠️ Always round the sample size up — rounding down would not meet the error requirement.
One-Sided Confidence Bounds
Note (all of in one tail), not .
5. Estimating the Mean — Unknown
Theorem — CI for , Unknown (normal population)
Derivation. Same as above, starting from (Chapter 11).
Worked Example
Fill volumes (mL, above 900) of 8 bottles: .
- , ; , ,
- ,
95% upper bound: .
Large-Sample Case
If and is unknown, replacing by and using is acceptable even if the population is not normal (CLT); and critical values are then nearly equal.
flowchart TD A["CI for μ"] --> B{"σ known?"} B -- Yes --> C["z interval:<br/>x̄ ± z(α/2)·σ/√n"] B -- No --> D{"n ≥ 30?"} D -- Yes --> E["z interval with s<br/>(or t — nearly equal)"] D -- No --> F{"Population<br/>≈ normal?"} F -- Yes --> G["t interval, v = n − 1:<br/>x̄ ± t(α/2)·s/√n"] F -- No --> H["Need more data or<br/>nonparametric methods"]
6. Standard Error of an Estimate
Definition — Standard Error
The standard error of an estimator is its standard deviation, . When it involves unknown parameters, they are replaced by estimates.
Most CIs have the form
Example. For the bottle data, the estimated standard error of is mL.
7. Prediction Intervals
Definition — Prediction Interval
An interval that, with probability , will contain a single future observation (not the mean).
Theorem — Prediction Interval for a New Observation (normal population)
Derivation. has mean and, since is independent of the sample,
Standardize and, if is unknown, replace it by to get a variable.
Worked Example
95% prediction interval for the next bottle:
Compare the 95% CI for the mean: .
| Confidence interval | Prediction interval | |
|---|---|---|
| Targets | the fixed parameter | one future random value |
| Width as | shrinks to 0 | shrinks to (never 0) |
| Factor |
⚠️ A prediction interval is always wider than the CI at the same confidence: individual values vary more than averages.