1. General Overview
Definition — Statistics
Statistics is the science of collecting, organizing, summarizing, analyzing and interpreting data in order to make decisions under uncertainty (variability).
Definition — Population and Sample
- Population: the entire collection of individuals/measurements of interest.
- Sample: a subset of the population that is actually observed.
| Term | Describes | Typical symbol |
|---|---|---|
| Parameter | a numerical property of the population (fixed, usually unknown) | |
| Statistic | a numerical property of the sample (computed, varies from sample to sample) |
Definition — Descriptive vs. Inferential Statistics
- Descriptive statistics: summarize the data at hand (tables, plots, mean, standard deviation).
- Inferential statistics: use a sample to draw conclusions about the population, with a stated level of uncertainty.
The Role of Probability
Probability and inference run in opposite directions:
flowchart LR P["Population<br/>(known model)"] -- "Probability:<br/>what samples are likely?" --> S["Sample<br/>(data)"] S -- "Statistical inference:<br/>what population produced this?" --> P
Example. A process is known to produce 5% defectives (population known). Probability asks: “How likely is it to see 3 or more defectives in a sample of 20?” Inference asks the reverse: “We saw 3 defectives in 20; is the 5% claim still believable?”
⚠️ A statistic is a random variable (it changes with the sample); a parameter is a constant. Never write when you computed .
2. Sampling Procedures; Collection of Data
Definition — Simple Random Sample (SRS)
A sample of size chosen so that every possible subset of size has the same chance of being selected.
The aim is to avoid bias: a systematic tendency to over- or under-represent part of the population.
Common Sampling Schemes
| Scheme | How it works | When useful |
|---|---|---|
| Simple random | Random selection from the whole list | Homogeneous population |
| Stratified | Split population into non-overlapping strata (e.g., by faculty), then take an SRS inside each | Strata differ strongly from each other |
| Cluster | Randomly select whole groups (clusters) and observe everyone in them | Cheap when units are naturally grouped |
| Systematic | Every -th unit after a random start | Production lines, ordered lists |
Definition — Experimental Design Terms
- Experimental units: the objects on which measurements are made.
- Treatment / factor: the condition deliberately changed by the experimenter.
- Control group: receives no treatment (or a placebo) for comparison.
- Randomization: assigning units to treatments at random, to average out unknown influences.
- Replication: applying each treatment to several units, to measure natural variability.
Example. To test whether a new solder alloy reduces joint failures, 40 circuit boards are randomly split: 20 get the new alloy (treatment), 20 the standard one (control). Randomization protects against e.g. all the “good” boards ending up in one group.
⚠️ Observational study ≠ designed experiment. Only a designed (randomized) experiment can support a cause-and-effect conclusion; observational data can show association only.
3. Measures of Location
Let the data be .
Definition — Sample Mean
Definition — Sample Median
Sort the data: .
Definition — Mode and Trimmed Mean
- Mode: the most frequent value.
- trimmed mean : remove the largest and smallest of the sorted data, then average the rest.
Property — Deviations from the Mean Sum to Zero
Proof.
(This is why the mean is the “balance point” of the data.)
Worked Example — Robustness
Data: ().
- Mean:
- Sorted: → median
- trimmed mean (drop one value from each end: 11 and 40):
The single value 40 pulls the mean up by almost 3 units; the median and trimmed mean barely move.
| Measure | Value | Sensitive to outliers? |
|---|---|---|
| Mean | 17.375 | Yes |
| Median | 14.5 | No |
| 12.5% trimmed mean | 14.67 | Much less |
⚠️ For skewed data (incomes, waiting times) the median is usually a better “typical value” than the mean.
4. Measures of Variability
Definition — Sample Range
Definition — Sample Variance and Standard Deviation
is the degrees of freedom: since the deviations must sum to zero, only of them are free.
Theorem — Computational Formula
Proof.
Definition — Quartiles and Interquartile Range
, , split the sorted data into four parts; .
Worked Example — Variance
Data: ().
- , median , mode ,
⚠️ Using instead of gives a biased estimate of (proved in Chapter 10). Calculators label the two as / — use for sample data.
⚠️ Variance is in squared units; the standard deviation is in the original units.
5. Discrete and Continuous Data
| Type | Values | Examples | Typical model |
|---|---|---|---|
| Discrete (count) | countable: | defectives per lot, packets lost | binomial, Poisson |
| Continuous (measurement) | any value in an interval | voltage, time to failure | normal, exponential |
| Categorical | labels | pass/fail, brand | proportions, contingency tables |
Categorical data are summarized with proportions: if of units are in a category, .
6. Statistical Modeling and Graphical Diagnostics
Definition — Statistical Model
A mathematical description of how data are generated, e.g. where is random error. Plots are used to check whether a model is reasonable.
Main Graphical Tools
| Plot | Shows | Use it to check |
|---|---|---|
| Dot plot | each observation on a line | small data, clusters |
| Stem-and-leaf | data split into stem | leaf; keeps actual values | shape + exact values |
| Histogram (frequency / relative frequency) | counts in class intervals | shape: symmetric, skewed, bimodal |
| Box-and-whisker plot | , whiskers, outliers | center, spread, skew, outliers |
| Scatter plot | pairs | relationship between two variables |
| Normal quantile plot | sorted data vs. normal quantiles (Chapter 11) | is the data roughly normal? |
Definition — Skewness (shape)
- Symmetric: mean ≈ median.
- Right (positively) skewed: long right tail, mean > median.
- Left (negatively) skewed: long left tail, mean < median.
Box-Plot Outlier Rule
An observation is a (suspected) outlier if it lies more than beyond a quartile:
Worked Example — Box Plot
Data from Section 3: .
Using linear-interpolation quartiles: , , , so .
Fences: and . The value → outlier. Whiskers extend to the most extreme non-outliers, 11 and 18.
⚠️ Different textbooks/software compute quartiles slightly differently (e.g. median-of-halves gives , ). Use the convention your instructor uses; conclusions rarely change.
7. General Types of Statistical Studies
| Study type | Researcher controls treatment? | Can show causation? | Example |
|---|---|---|---|
| Designed experiment | Yes, with randomization | Yes | Randomly assign boards to two solder alloys |
| Observational study | No, just observes | No (association only) | Compare failure rates of laptops of users who do/don’t use cooling pads |
| Retrospective study | No, uses historical records | No | Analyze last year’s production logs |
Definition — Confounding
Two factors are confounded when their effects on the response cannot be separated (e.g., the users who buy cooling pads may also be heavier users).
⚠️ “Correlation does not imply causation”: an observational association can be explained by a hidden (lurking) variable.