1. Mean of a Random Variable

Definition — Expected Value (Mean)


is the long-run average of over many repetitions — the “center of mass” of the distribution.

Theorem — Expectation of a Function


and for two variables, or .

(You don’t need the distribution of — weight by directly.)

Worked Example — Discrete

= heads in 3 coin tosses, on .

A game pays dollars: dollars.

Worked Example — Continuous

on (from Chapter 3):


2. Variance

Definition — Variance and Standard Deviation

Theorem — Computational Formula

Proof.

Worked Examples

  • Coins: , so .
  • Continuous: , so .

⚠️ in general; their difference is the variance, which is never negative. A negative “variance” means an arithmetic error.


3. Covariance

Definition — Covariance

  • : large tends to go with large
  • : large tends to go with small

Theorem — Computational Formula


(Proof: expand the product exactly as for the variance.)

Theorem — Independence ⇒ Zero Covariance

If and are independent, and so .

Proof (continuous). .

⚠️ The converse is false. If is uniform on and , then , so , yet is completely determined by .

Worked Example — Discrete Joint Table (from Chapter 4)

0.100.200.100.40
0.150.250.050.45
0.050.050.050.15
0.300.500.201
  • ,
  • ,

4. Correlation Coefficient

Definition — Correlation Coefficient


is unit-free and measures the strength of the linear relationship. iff exactly (with probability 1).

Theorem —

Proof. For any real , . A quadratic in that is never negative has discriminant : , i.e. .

Worked Examples

  • Table: — a very weak negative linear relation.
  • Density on the unit square: , , so ; with , .
Simulated pairs with sample correlation r = -0.90 (strong negative linear relation) y x 2.2 3.7 5.2 6.8 8.3

5. Means and Variances of Linear Combinations

Theorem — Linearity of Expectation


Theorem — Variance of a Linear Function

Proof. , so .

(Adding a constant shifts the distribution but does not change its spread.)

Theorem — Variance of a Linear Combination


If are independent: . More generally, for independent :

Proof. ; expand the square and take expectations term by term.

Worked Example

With the table above, :

⚠️ Variances add even for a difference: for independent , (not minus). Subtracting a random quantity adds uncertainty.


6. Chebyshev’s Theorem

Theorem — Chebyshev’s Inequality

For any RV with mean and standard deviation , and any :

Proof (continuous).

Divide by .

Chebyshev guarantees at leastNormal distribution actually gives
1.555.6%86.6%
275%95.4%
388.9%99.7%

Worked Example

has , (distribution unknown). Bound :
and , so and .

⚠️ Chebyshev gives a lower bound valid for every distribution, so it is usually conservative. It is useless for .