1. Mean of a Random Variable
Definition — Expected Value (Mean)
is the long-run average of over many repetitions — the “center of mass” of the distribution.
Theorem — Expectation of a Function
and for two variables, or .
(You don’t need the distribution of — weight by directly.)
Worked Example — Discrete
= heads in 3 coin tosses, on .
A game pays dollars: dollars.
Worked Example — Continuous
on (from Chapter 3):
2. Variance
Definition — Variance and Standard Deviation
Theorem — Computational Formula
Proof.
Worked Examples
- Coins: , so .
- Continuous: , so .
⚠️ in general; their difference is the variance, which is never negative. A negative “variance” means an arithmetic error.
3. Covariance
Definition — Covariance
- : large tends to go with large
- : large tends to go with small
Theorem — Computational Formula
(Proof: expand the product exactly as for the variance.)
Theorem — Independence ⇒ Zero Covariance
If and are independent, and so .
Proof (continuous). .
⚠️ The converse is false. If is uniform on and , then , so , yet is completely determined by .
Worked Example — Discrete Joint Table (from Chapter 4)
| 0.10 | 0.20 | 0.10 | 0.40 | |
| 0.15 | 0.25 | 0.05 | 0.45 | |
| 0.05 | 0.05 | 0.05 | 0.15 | |
| 0.30 | 0.50 | 0.20 | 1 |
- ,
- ,
4. Correlation Coefficient
Definition — Correlation Coefficient
is unit-free and measures the strength of the linear relationship. iff exactly (with probability 1).
Theorem —
Proof. For any real , . A quadratic in that is never negative has discriminant : , i.e. .
Worked Examples
- Table: — a very weak negative linear relation.
- Density on the unit square: , , so ; with , .
5. Means and Variances of Linear Combinations
Theorem — Linearity of Expectation
Theorem — Variance of a Linear Function
Proof. , so .
(Adding a constant shifts the distribution but does not change its spread.)
Theorem — Variance of a Linear Combination
If are independent: . More generally, for independent :
Proof. ; expand the square and take expectations term by term.
Worked Example
With the table above, :
⚠️ Variances add even for a difference: for independent , (not minus). Subtracting a random quantity adds uncertainty.
6. Chebyshev’s Theorem
Theorem — Chebyshev’s Inequality
For any RV with mean and standard deviation , and any :
Proof (continuous).
Divide by .
| Chebyshev guarantees at least | Normal distribution actually gives | |
|---|---|---|
| 1.5 | 55.6% | 86.6% |
| 2 | 75% | 95.4% |
| 3 | 88.9% | 99.7% |
Worked Example
has , (distribution unknown). Bound :
and , so and .
⚠️ Chebyshev gives a lower bound valid for every distribution, so it is usually conservative. It is useless for .