1. Statistical Inference

Definition — Statistical Inference

Drawing conclusions about population parameters from sample data. Two main branches:

BranchQuestionOutputChapters
Estimation”What is the value of ?“a point estimate or an interval12–13
Hypothesis testing”Is believable?“accept / reject a claim14

Classical vs. Bayesian

The classical (frequentist) approach treats parameters as fixed unknown constants and bases inference only on the sample. The Bayesian approach also uses prior beliefs about the parameter. This course uses the classical approach.


2. Methods of Estimation: Point Estimation

Definition — Estimator and Estimate

An estimator is a statistic used to estimate a parameter (a random variable). Its observed value is the point estimate.
Examples: estimates ; estimates ; estimates .

Definition — Unbiased Estimator

is unbiased for if . The bias is .

  • is unbiased for ; is unbiased for (proved in Chapter 10).
  • is biased: .

Definition — Efficiency and MVUE

Among unbiased estimators, the one with the smallest variance is the most efficient — the minimum variance unbiased estimator (MVUE).
For a normal population, both and the sample median are unbiased for , but , so is more efficient.

Two unbiased estimators of μ: X̄ (black) is more efficient than the median (amber, wider) μ-3σ/√n μ-2σ/√n μ-1σ/√n μ μ+1σ/√n μ+2σ/√n μ+3σ/√n

Definition — Maximum Likelihood Estimation

Given data , the likelihood is . The maximum likelihood estimator is the value that maximizes (usually by setting ).

Example. Exponential lifetimes with mean : . Setting gives .


3. Methods of Estimation: Interval Estimation

Definition — Confidence Interval

An interval computed from the sample such that, before sampling,

is a confidence interval; is the confidence coefficient.

Interpretation

If we repeated the sampling many times and built an interval each time, about of the intervals would contain the true .

⚠️ Once computed, a specific interval like either contains or not. It is wrong to say “there is a 95% probability that is in ”; the 95% describes the method.

Trade-offs

  • Higher confidence → wider interval.
  • Larger → narrower interval (width ).

4. Estimating the Mean — Known

Theorem — CI for , Known

If is the mean of a random sample of size from a population with known (normal population, or large):

Derivation. , so

Multiply through by , subtract and multiply by (which flips the inequalities) to isolate .

Confidence90%95%99%
1.6451.9602.576

Worked Example

Zinc concentration from locations: g/mL; known. .

  • 95%:
  • 99%:
20 simulated 95% CIs (n = 36, σ = 0.3): about 1 in 20 misses μ true μ = 2.6 2.31 2.45 2.60 2.75 2.89

Theorem — Error Bound and Sample Size

With confidence, the error does not exceed . To guarantee a maximum error :

Example. For error at most 0.05 with 95% confidence: .

⚠️ Always round the sample size up — rounding down would not meet the error requirement.

One-Sided Confidence Bounds


Note (all of in one tail), not .


5. Estimating the Mean — Unknown

Theorem — CI for , Unknown (normal population)

Derivation. Same as above, starting from (Chapter 11).

Worked Example

Fill volumes (mL, above 900) of 8 bottles: .

  • , ; , ,
  • ,

95% upper bound: .

Large-Sample Case

If and is unknown, replacing by and using is acceptable even if the population is not normal (CLT); and critical values are then nearly equal.

flowchart TD
    A["CI for μ"] --> B{"σ known?"}
    B -- Yes --> C["z interval:<br/>x̄ ± z(α/2)·σ/√n"]
    B -- No --> D{"n ≥ 30?"}
    D -- Yes --> E["z interval with s<br/>(or t — nearly equal)"]
    D -- No --> F{"Population<br/>≈ normal?"}
    F -- Yes --> G["t interval, v = n − 1:<br/>x̄ ± t(α/2)·s/√n"]
    F -- No --> H["Need more data or<br/>nonparametric methods"]

6. Standard Error of an Estimate

Definition — Standard Error

The standard error of an estimator is its standard deviation, . When it involves unknown parameters, they are replaced by estimates.

Most CIs have the form

Example. For the bottle data, the estimated standard error of is mL.


7. Prediction Intervals

Definition — Prediction Interval

An interval that, with probability , will contain a single future observation (not the mean).

Theorem — Prediction Interval for a New Observation (normal population)

Derivation. has mean and, since is independent of the sample,

Standardize and, if is unknown, replace it by to get a variable.

Worked Example

95% prediction interval for the next bottle:

Compare the 95% CI for the mean: .

Confidence intervalPrediction interval
Targetsthe fixed parameter one future random value
Width as shrinks to 0shrinks to (never 0)
Factor

⚠️ A prediction interval is always wider than the CI at the same confidence: individual values vary more than averages.