1. Random Experiments, Sample Spaces and Events

Definition — Random Experiment

A process whose outcome cannot be predicted with certainty, but whose set of possible outcomes is known (toss a coin, measure a resistor, count packets).

Definition — Sample Space

The sample space is the set of all possible outcomes.

  • Discrete: ,
  • Continuous: (lifetime of a component)

Definition — Event

An event is a subset of . The null event contains no outcomes.

Set Operations on Events

OperationNotationMeaning
Complement does not occur
Intersectionboth and occur
Union or (or both) occur
Mutually exclusivecannot occur together

Theorem — De Morgan’s Laws

Example. Two dice are rolled. = “sum is 7”, = “first die shows 1”.
,


2. Counting Techniques

Theorem — Multiplication Rule

If operation 1 can be done in ways and, for each, operation 2 in ways, …, then the operations together can be done in

Definition — Permutations (order matters)

Number of ordered arrangements of objects chosen from distinct objects:

Proof. First position: choices, second: , …, -th: . By the multiplication rule the product is .

Theorem — Circular Permutations and Permutations with Repetition

  • distinct objects around a circle:
  • objects of which alike, alike, …, alike:

Definition — Combinations (order does not matter)

Proof. Each unordered set of objects can be ordered in ways, so .

Worked Examples

QuestionToolAnswer
Elect president, VP, secretary from 10 peoplepermutation
Choose a 3-person committee from 10combination
Distinct arrangements of the letters of MISSISSIPPIrepetition
5-card poker handscombination

⚠️ Ask yourself: “Does swapping two chosen items give a different outcome?” Yes → permutation; No → combination.


3. Probability of an Event

Definition — Axioms of Probability

For every event :

  1. ,
  2. If are mutually exclusive,

Theorem — Equally Likely Outcomes

If has equally likely outcomes and contains of them,

Worked Example — Full House

Probability of a full house (3 of one rank + 2 of another) in a 5-card hand:


4. Additive Rules

Theorem — Addition Rule (Inclusion–Exclusion)

Proof. Write (disjoint) and (disjoint). Axiom 3 gives and . Substitute.

Corollary — Three Events

Theorem — Complement Rule

Proof. and are disjoint and , so .

Worked Example — Birthday Problem

Probability that at least two of 23 people share a birthday (365 equally likely days):

⚠️ “At least one” problems are almost always easiest through the complement.


5. Conditional Probability

Definition — Conditional Probability

The probability of given that has occurred ():

Conditioning on shrinks the sample space to .

Theorem — Multiplicative (Product) Rule


and for several events

Example. A box holds 4 good and 2 defective fuses. Two are drawn without replacement.
.


6. Independence

Definition — Independent Events

and are independent iff

Theorem — Complements of Independent Events

If and are independent, so are and (and and ).

Proof. .

Worked Example — System Reliability

Components work independently with the reliabilities shown.

flowchart LR
    IN((in)) --> A["A: 0.95"]
    A --> B["B: 0.90"]
    A --> C["C: 0.90"]
    B --> D["D: 0.98"]
    C --> D
    D --> OUT((out))
  • B and C in parallel: works unless both fail →
  • A, (B‖C), D in series: all must work →

⚠️ Independent ≠ mutually exclusive. If and are mutually exclusive, then , so they are dependent (knowing happened tells you did not).


7. Total Probability and Bayes’ Rule

Theorem — Law of Total Probability

If partition (mutually exclusive, union , each ), then for any event

Proof. , a disjoint union. Apply Axiom 3 then the product rule to each term.

Theorem — Bayes’ Rule

Proof. ; write the numerator with the product rule and the denominator with total probability.

TermName
prior probability
likelihood
posterior probability

Worked Example — Which Machine Made the Defect?

Machines A, B, C produce 50%, 30%, 20% of the output, with defect rates 1%, 2%, 3%.

flowchart LR
    S((start)) -- "0.50" --> A[A]
    S -- "0.30" --> B[B]
    S -- "0.20" --> C[C]
    A -- "0.01" --> AD["D: 0.005"]
    A -- "0.99" --> AG[good]
    B -- "0.02" --> BD["D: 0.006"]
    B -- "0.98" --> BG[good]
    C -- "0.03" --> CD["D: 0.006"]
    C -- "0.97" --> CG[good]


Even though C makes only 20% of the parts, it is (tied for) the most likely source of a defective.

Worked Example — Diagnostic Test (base-rate effect)

A disease has prevalence 1%. The test has sensitivity and specificity .

⚠️ . With a rare condition, most positives are false positives even for a good test. Always identify which event is being conditioned on.