1. General Overview

Definition — Statistics

Statistics is the science of collecting, organizing, summarizing, analyzing and interpreting data in order to make decisions under uncertainty (variability).

Definition — Population and Sample

  • Population: the entire collection of individuals/measurements of interest.
  • Sample: a subset of the population that is actually observed.
TermDescribesTypical symbol
Parametera numerical property of the population (fixed, usually unknown)
Statistica numerical property of the sample (computed, varies from sample to sample)

Definition — Descriptive vs. Inferential Statistics

  • Descriptive statistics: summarize the data at hand (tables, plots, mean, standard deviation).
  • Inferential statistics: use a sample to draw conclusions about the population, with a stated level of uncertainty.

The Role of Probability

Probability and inference run in opposite directions:

flowchart LR
    P["Population<br/>(known model)"] -- "Probability:<br/>what samples are likely?" --> S["Sample<br/>(data)"]
    S -- "Statistical inference:<br/>what population produced this?" --> P

Example. A process is known to produce 5% defectives (population known). Probability asks: “How likely is it to see 3 or more defectives in a sample of 20?” Inference asks the reverse: “We saw 3 defectives in 20; is the 5% claim still believable?”

⚠️ A statistic is a random variable (it changes with the sample); a parameter is a constant. Never write when you computed .


2. Sampling Procedures; Collection of Data

Definition — Simple Random Sample (SRS)

A sample of size chosen so that every possible subset of size has the same chance of being selected.

The aim is to avoid bias: a systematic tendency to over- or under-represent part of the population.

Common Sampling Schemes

SchemeHow it worksWhen useful
Simple randomRandom selection from the whole listHomogeneous population
StratifiedSplit population into non-overlapping strata (e.g., by faculty), then take an SRS inside eachStrata differ strongly from each other
ClusterRandomly select whole groups (clusters) and observe everyone in themCheap when units are naturally grouped
SystematicEvery -th unit after a random startProduction lines, ordered lists

Definition — Experimental Design Terms

  • Experimental units: the objects on which measurements are made.
  • Treatment / factor: the condition deliberately changed by the experimenter.
  • Control group: receives no treatment (or a placebo) for comparison.
  • Randomization: assigning units to treatments at random, to average out unknown influences.
  • Replication: applying each treatment to several units, to measure natural variability.

Example. To test whether a new solder alloy reduces joint failures, 40 circuit boards are randomly split: 20 get the new alloy (treatment), 20 the standard one (control). Randomization protects against e.g. all the “good” boards ending up in one group.

⚠️ Observational study ≠ designed experiment. Only a designed (randomized) experiment can support a cause-and-effect conclusion; observational data can show association only.


3. Measures of Location

Let the data be .

Definition — Sample Mean

Definition — Sample Median

Sort the data: .

Definition — Mode and Trimmed Mean

  • Mode: the most frequent value.
  • trimmed mean : remove the largest and smallest of the sorted data, then average the rest.

Property — Deviations from the Mean Sum to Zero

Proof.

(This is why the mean is the “balance point” of the data.)

Worked Example — Robustness

Data: ().

  • Mean:
  • Sorted: → median
  • trimmed mean (drop one value from each end: 11 and 40):

The single value 40 pulls the mean up by almost 3 units; the median and trimmed mean barely move.

MeasureValueSensitive to outliers?
Mean17.375Yes
Median14.5No
12.5% trimmed mean14.67Much less

⚠️ For skewed data (incomes, waiting times) the median is usually a better “typical value” than the mean.


4. Measures of Variability

Definition — Sample Range

Definition — Sample Variance and Standard Deviation

is the degrees of freedom: since the deviations must sum to zero, only of them are free.

Theorem — Computational Formula

Proof.

Definition — Quartiles and Interquartile Range

, , split the sorted data into four parts; .

Worked Example — Variance

Data: ().

  • , median , mode ,

⚠️ Using instead of gives a biased estimate of (proved in Chapter 10). Calculators label the two as / — use for sample data.

⚠️ Variance is in squared units; the standard deviation is in the original units.


5. Discrete and Continuous Data

TypeValuesExamplesTypical model
Discrete (count)countable: defectives per lot, packets lostbinomial, Poisson
Continuous (measurement)any value in an intervalvoltage, time to failurenormal, exponential
Categoricallabelspass/fail, brandproportions, contingency tables

Categorical data are summarized with proportions: if of units are in a category, .


6. Statistical Modeling and Graphical Diagnostics

Definition — Statistical Model

A mathematical description of how data are generated, e.g. where is random error. Plots are used to check whether a model is reasonable.

Main Graphical Tools

PlotShowsUse it to check
Dot ploteach observation on a linesmall data, clusters
Stem-and-leafdata split into stem | leaf; keeps actual valuesshape + exact values
Histogram (frequency / relative frequency)counts in class intervalsshape: symmetric, skewed, bimodal
Box-and-whisker plot, whiskers, outlierscenter, spread, skew, outliers
Scatter plotpairs relationship between two variables
Normal quantile plotsorted data vs. normal quantiles (Chapter 11)is the data roughly normal?

Definition — Skewness (shape)

  • Symmetric: mean ≈ median.
  • Right (positively) skewed: long right tail, mean > median.
  • Left (negatively) skewed: long left tail, mean < median.

Box-Plot Outlier Rule

An observation is a (suspected) outlier if it lies more than beyond a quartile:

Worked Example — Box Plot

Data from Section 3: .
Using linear-interpolation quartiles: , , , so .

Fences: and . The value → outlier. Whiskers extend to the most extreme non-outliers, 11 and 18.

Box plot of 11, 12, 13, 14, 15, 16, 18, 40 outlier 40 Q1=12.75 median=14.5 Q3=16.5 10 15 20 25 30 35 40

⚠️ Different textbooks/software compute quartiles slightly differently (e.g. median-of-halves gives , ). Use the convention your instructor uses; conclusions rarely change.


7. General Types of Statistical Studies

Study typeResearcher controls treatment?Can show causation?Example
Designed experimentYes, with randomizationYesRandomly assign boards to two solder alloys
Observational studyNo, just observesNo (association only)Compare failure rates of laptops of users who do/don’t use cooling pads
Retrospective studyNo, uses historical recordsNoAnalyze last year’s production logs

Definition — Confounding

Two factors are confounded when their effects on the response cannot be separated (e.g., the users who buy cooling pads may also be heavier users).

⚠️ “Correlation does not imply causation”: an observational association can be explained by a hidden (lurking) variable.