Part 1 · 3 — Experimental Design Basics

Part 1 · 3 — Experimental Design Basics#

Designs we can run#

  • A/B, switchback, staggered rollout

  • Before/after (with controls) vs simple pre/post

  • Power & sample size (intuitions first)

Threats to validity#

  • Selection, drift, seasonality, interference

  • Novelty & adoption effects

Minimal plan#

  • Assignment, randomization, units

  • Primary metric, guardrails

  • Duration and decision rule

Coming next

Monte Carlo risk bands → Bayes triage → Survival time-to-exploit.

Contents