1. Random Sampling
Definition — Random Sample
is a random sample of size from a population with distribution if the are independent and each has the same distribution (i.i.d.). Their joint density is
Definition — Statistic
A statistic is any function of the random sample that does not depend on unknown parameters (e.g. , , ). Since the sample is random, a statistic is itself a random variable.
Definition — Sampling Distribution
The probability distribution of a statistic is its sampling distribution. It describes how the statistic varies over all possible samples of size .
flowchart LR P["Population<br/>mean μ, SD σ"] --> S1["Sample 1 → x̄₁"] P --> S2["Sample 2 → x̄₂"] P --> S3["Sample 3 → x̄₃"] P --> Sk["… many samples"] S1 --> D["Distribution of the x̄ values<br/>= sampling distribution of X̄"] S2 --> D S3 --> D Sk --> D
2. The Sample Statistics
Definitions — Important Statistics
| Statistic | Formula |
|---|---|
| Sample mean | |
| Sample median | middle of the ordered sample |
| Sample variance | |
| Sample standard deviation | |
| Sample range |
Theorem — is Unbiased for
Proof. Use (expand ; the cross term vanishes since ). Taking expectations, and (Section 3):
Dividing by gives .
⚠️ This is why we divide by : dividing by would systematically underestimate by the factor .
3. Sampling Distribution of the Mean
Theorem — Mean and Variance of
For a random sample from any population with mean and variance :
Proof. . By independence, .
is the standard error of the mean. Quadrupling halves it.
Theorem — Normal Population
If the population is normal, exactly, for every (Chapter 9: linear combinations of independent normals are normal).
4. The Central Limit Theorem
Theorem — Central Limit Theorem (CLT)
If is the mean of a random sample of size from any population with mean and finite variance , then
Proof sketch (via MGFs, assuming the MGF exists). Let (mean 0, variance 1), so and
(Taylor expansion of using , , then .) is the MGF.
Rule of Thumb
- Population normal → exact for any .
- Population roughly symmetric → good for or so.
- Generally is adequate unless the population is very skewed.
Worked Example — Normal Population
Bulb lifetimes have h, h. A sample of :
Worked Example — Skewed Population
Service times are exponential with mean 4 min (so ). For customers:
(Exact answer, using Gamma: — the CLT is close but slightly off because the population is very skewed.)
⚠️ The CLT is about the distribution of the sample mean, not the data. A large sample from a skewed population is still skewed; only becomes approximately normal.
5. Sampling Distribution of the Difference Between Two Means
Theorem — Difference of Two Means
Independent samples of sizes , from populations with means and variances :
(exact if both populations are normal).
Why: and, by independence, .
Worked Example
Two paint brands’ drying times: A has h, ; B has h, . Samples , . Probability the A-sample mean exceeds the B-sample mean by more than 1 h:
6. Sampling Distribution of the Population Proportion
Theorem — Sample Proportion
If and :
and for large (with , ):
Why: is the mean of Bernoulli variables, each with mean and variance ; apply Section 3 and the CLT.
Worked Example
30% of users have enabled 2-factor authentication. In a random sample of 200 users:
7. Sampling Distribution of the Difference Between Two Proportions
Theorem — Difference of Two Proportions
For independent large samples:
Worked Example
Defect rates: line 1 has , line 2 has . Samples , . Probability that the samples show line 1 better than line 2 ():
Summary Table
| Statistic | Mean | Standard error | Approx. distribution |
|---|---|---|---|
| Normal (CLT) | |||
| Normal | |||
| Normal | |||
| Normal |
⚠️ These formulas use the known (or true ). When must be estimated by from a small sample, is replaced by the statistic (Chapter 11).