Sampling, Confidence & Hypothesis Tests

Draw conclusions about a population from a sample. Learn sampling distributions and the central limit theorem, standard error, confidence intervals (z and t), hypothesis testing, p-values, Type I and II errors, power and sample size, and common tests (t, paired, two-sample, chi-square).

Did the new process really improve strength, or is the difference just noise? Inference quantifies how confident we can be, so decisions are based on evidence rather than luck.

An analogy

A hypothesis test is like a court trial: the null hypothesis is 'innocent until proven guilty', and the p-value measures how surprising the evidence would be if it were innocent.

Sampling distributions

Sample means vary from sample to sample; their spread is σ/√n. Quadrupling n halves the uncertainty. The CLT makes x̄ approximately normal regardless of the population shape.

Confidence intervals

A 95 % CI means the method captures the true mean in 95 % of repeated samples. Use t (not z) for small samples with unknown σ.

Hypothesis testing

State H₀ and H₁, choose α (often 0.05), compute the statistic and p-value, and decide. p < α: reject H₀. A non-significant result is not proof of no effect — check power.

Errors and power

Type I: false alarm (reject true H₀); Type II: missed effect. Power increases with sample size, effect size and lower noise. Statistical significance ≠ practical significance — report effect sizes and CIs.

Formulas

• CI for mean (σ unknown): x̄ ± t_{α/2, n−1} s/√n
• One-sample t: t = (x̄ − μ₀)/(s/√n)
• Sample size: n = (z_{α/2} σ/E)²
• CLT: x̄ ≈ N(μ, σ²/n) for large n

Worked example: 95 % CI

Given: n = 10, x̄ = 12.10, s = 0.279; t₀.₉₇₅,₉ = 2.262

Find: CI.

• SE = 0.279/√10 = 0.0882
• 12.10 ± 2.262 × 0.0882 = 12.10 ± 0.20

Answer: [11.90, 12.30] mm

Worked example: Sample size

Given: σ = 0.27 mm, margin ± 0.1 mm, 95 %

Find: n.

• n = (1.96 × 0.27/0.1)² = 28.0

Answer: n = 28 (round up)

Common mistakes

• Wrong: p > 0.05 proves no effect. — Right: It may be underpowered — report CIs.
• Wrong: Running many tests without correction. — Right: Multiple comparisons inflate false alarms.

Applications

• Process changes: Did a modification improve performance?
• Acceptance testing: Does the product meet the spec mean?
• A/B testing: Comparing two designs statistically.

Key takeaways

• SE = σ/√n; CLT.
• CI: x̄ ± t·s/√n.
• p-value vs α; Type I/II errors.
• Plan sample size for power.

Going further

• ANOVA: Comparing several means.
• Non-parametric tests: Wilcoxon, Mann–Whitney.
• Bayesian inference: Posterior probabilities instead of p-values.