1. Random Experiments, Sample Spaces and Events
Definition — Random Experiment
A process whose outcome cannot be predicted with certainty, but whose set of possible outcomes is known (toss a coin, measure a resistor, count packets).
Definition — Sample Space
The sample space is the set of all possible outcomes.
- Discrete: ,
- Continuous: (lifetime of a component)
Definition — Event
An event is a subset of . The null event contains no outcomes.
Set Operations on Events
| Operation | Notation | Meaning |
|---|---|---|
| Complement | does not occur | |
| Intersection | both and occur | |
| Union | or (or both) occur | |
| Mutually exclusive | cannot occur together |
Theorem — De Morgan’s Laws
Example. Two dice are rolled. = “sum is 7”, = “first die shows 1”.
,
2. Counting Techniques
Theorem — Multiplication Rule
If operation 1 can be done in ways and, for each, operation 2 in ways, …, then the operations together can be done in
Definition — Permutations (order matters)
Number of ordered arrangements of objects chosen from distinct objects:
Proof. First position: choices, second: , …, -th: . By the multiplication rule the product is .
Theorem — Circular Permutations and Permutations with Repetition
- distinct objects around a circle:
- objects of which alike, alike, …, alike:
Definition — Combinations (order does not matter)
Proof. Each unordered set of objects can be ordered in ways, so .
Worked Examples
| Question | Tool | Answer |
|---|---|---|
| Elect president, VP, secretary from 10 people | permutation | |
| Choose a 3-person committee from 10 | combination | |
| Distinct arrangements of the letters of MISSISSIPPI | repetition | |
| 5-card poker hands | combination |
⚠️ Ask yourself: “Does swapping two chosen items give a different outcome?” Yes → permutation; No → combination.
3. Probability of an Event
Definition — Axioms of Probability
For every event :
- ,
- If are mutually exclusive,
Theorem — Equally Likely Outcomes
If has equally likely outcomes and contains of them,
Worked Example — Full House
Probability of a full house (3 of one rank + 2 of another) in a 5-card hand:
4. Additive Rules
Theorem — Addition Rule (Inclusion–Exclusion)
Proof. Write (disjoint) and (disjoint). Axiom 3 gives and . Substitute.
Corollary — Three Events
Theorem — Complement Rule
Proof. and are disjoint and , so .
Worked Example — Birthday Problem
Probability that at least two of 23 people share a birthday (365 equally likely days):
⚠️ “At least one” problems are almost always easiest through the complement.
5. Conditional Probability
Definition — Conditional Probability
The probability of given that has occurred ():
Conditioning on shrinks the sample space to .
Theorem — Multiplicative (Product) Rule
and for several events
Example. A box holds 4 good and 2 defective fuses. Two are drawn without replacement.
.
6. Independence
Definition — Independent Events
and are independent iff
Theorem — Complements of Independent Events
If and are independent, so are and (and and ).
Proof. .
Worked Example — System Reliability
Components work independently with the reliabilities shown.
flowchart LR IN((in)) --> A["A: 0.95"] A --> B["B: 0.90"] A --> C["C: 0.90"] B --> D["D: 0.98"] C --> D D --> OUT((out))
- B and C in parallel: works unless both fail →
- A, (B‖C), D in series: all must work →
⚠️ Independent ≠ mutually exclusive. If and are mutually exclusive, then , so they are dependent (knowing happened tells you did not).
7. Total Probability and Bayes’ Rule
Theorem — Law of Total Probability
If partition (mutually exclusive, union , each ), then for any event
Proof. , a disjoint union. Apply Axiom 3 then the product rule to each term.
Theorem — Bayes’ Rule
Proof. ; write the numerator with the product rule and the denominator with total probability.
| Term | Name |
|---|---|
| prior probability | |
| likelihood | |
| posterior probability |
Worked Example — Which Machine Made the Defect?
Machines A, B, C produce 50%, 30%, 20% of the output, with defect rates 1%, 2%, 3%.
flowchart LR S((start)) -- "0.50" --> A[A] S -- "0.30" --> B[B] S -- "0.20" --> C[C] A -- "0.01" --> AD["D: 0.005"] A -- "0.99" --> AG[good] B -- "0.02" --> BD["D: 0.006"] B -- "0.98" --> BG[good] C -- "0.03" --> CD["D: 0.006"] C -- "0.97" --> CG[good]
Even though C makes only 20% of the parts, it is (tied for) the most likely source of a defective.
Worked Example — Diagnostic Test (base-rate effect)
A disease has prevalence 1%. The test has sensitivity and specificity .
⚠️ . With a rare condition, most positives are false positives even for a good test. Always identify which event is being conditioned on.