1. Random Sampling

Definition — Random Sample

is a random sample of size from a population with distribution if the are independent and each has the same distribution (i.i.d.). Their joint density is

Definition — Statistic

A statistic is any function of the random sample that does not depend on unknown parameters (e.g. , , ). Since the sample is random, a statistic is itself a random variable.

Definition — Sampling Distribution

The probability distribution of a statistic is its sampling distribution. It describes how the statistic varies over all possible samples of size .

flowchart LR
    P["Population<br/>mean μ, SD σ"] --> S1["Sample 1 → x̄₁"]
    P --> S2["Sample 2 → x̄₂"]
    P --> S3["Sample 3 → x̄₃"]
    P --> Sk["… many samples"]
    S1 --> D["Distribution of the x̄ values<br/>= sampling distribution of X̄"]
    S2 --> D
    S3 --> D
    Sk --> D

2. The Sample Statistics

Definitions — Important Statistics

StatisticFormula
Sample mean
Sample medianmiddle of the ordered sample
Sample variance
Sample standard deviation
Sample range

Theorem — is Unbiased for

Proof. Use (expand ; the cross term vanishes since ). Taking expectations, and (Section 3):

Dividing by gives .

⚠️ This is why we divide by : dividing by would systematically underestimate by the factor .


3. Sampling Distribution of the Mean

Theorem — Mean and Variance of

For a random sample from any population with mean and variance :

Proof. . By independence, .

is the standard error of the mean. Quadrupling halves it.

Theorem — Normal Population

If the population is normal, exactly, for every (Chapter 9: linear combinations of independent normals are normal).


4. The Central Limit Theorem

Theorem — Central Limit Theorem (CLT)

If is the mean of a random sample of size from any population with mean and finite variance , then

Proof sketch (via MGFs, assuming the MGF exists). Let (mean 0, variance 1), so and

(Taylor expansion of using , , then .) is the MGF.

Rule of Thumb

  • Population normal → exact for any .
  • Population roughly symmetric → good for or so.
  • Generally is adequate unless the population is very skewed.
CLT: distribution of X̄ from an exponential(mean 1): n = 1 black, n = 4 blue, n = 16 amber μ = 1 0 0.5 1 1.5 2 2.5 3 3.5

Worked Example — Normal Population

Bulb lifetimes have h, h. A sample of :

Worked Example — Skewed Population

Service times are exponential with mean 4 min (so ). For customers:

(Exact answer, using Gamma: — the CLT is close but slightly off because the population is very skewed.)

⚠️ The CLT is about the distribution of the sample mean, not the data. A large sample from a skewed population is still skewed; only becomes approximately normal.


5. Sampling Distribution of the Difference Between Two Means

Theorem — Difference of Two Means

Independent samples of sizes , from populations with means and variances :

(exact if both populations are normal).

Why: and, by independence, .

Worked Example

Two paint brands’ drying times: A has h, ; B has h, . Samples , . Probability the A-sample mean exceeds the B-sample mean by more than 1 h:


6. Sampling Distribution of the Population Proportion

Theorem — Sample Proportion

If and :

and for large (with , ):

Why: is the mean of Bernoulli variables, each with mean and variance ; apply Section 3 and the CLT.

Worked Example

30% of users have enabled 2-factor authentication. In a random sample of 200 users:


7. Sampling Distribution of the Difference Between Two Proportions

Theorem — Difference of Two Proportions

For independent large samples:

Worked Example

Defect rates: line 1 has , line 2 has . Samples , . Probability that the samples show line 1 better than line 2 ():

Summary Table

StatisticMeanStandard errorApprox. distribution
Normal (CLT)
Normal
Normal
Normal

⚠️ These formulas use the known (or true ). When must be estimated by from a small sample, is replaced by the statistic (Chapter 11).