Problem statement:
I had to prepare teaching material for a class showing them different types of statistical test in data analysis.
So, I broke the problem down in stages and asked question about each stage.
Prompt 1:
You are a statistician and data analyst. I am planning to teach following subject in statistical test:
t-test, Chi-Square Test, ANOVA, Pearson Correlation, Linear Regression, Logistic Regression, Shapiro-Wilk Test, Levene’s Test, Kolmogorov-Smirnov Test, Mann-Whitney U Test, Kruskal wallis test, Kolmogorov-Smirnov (KS) Test, non parametric test.
Tell me the order in which I should teach.
Also tell me which topic i should also cover that is not present above and could be relevant to industry.
DO NOT give me any code right now. I just want to know the order and the topics I should cover.
Give the output in bullet form
OUTPUT:
Stage 1: Foundations (add this — currently missing)
Before any test, cover:
- Hypothesis testing basics: null/alternative hypothesis, p-value, significance level (α), Type I/Type II errors, confidence intervals
- Types of data: nominal, ordinal, interval, ratio (this determines which test applies later)
- Parametric vs non-parametric tests: what the distinction means and why it matters
Without this, everything downstream feels like disconnected recipes rather than a coherent framework.
Stage 2: Assumption-checking tests
Teach these next since students need them to decide which test to use in Stage 3 and 4:
- Shapiro-Wilk Test (normality)
- Kolmogorov-Smirnov Test (normality / distribution comparison)
- Levene’s Test (equality of variances)
Stage 3: Comparing means — parametric
- t-test (one-sample, independent samples, paired samples)
- ANOVA (one-way, then two-way if time permits)
- Add: Post-hoc tests (Tukey HSD) — students will ask “ANOVA says groups differ, but which ones?”
Stage 4: Comparing means — non-parametric alternatives
Taught right after their parametric counterparts so students see the mapping clearly:
- Mann-Whitney U Test (non-parametric alternative to independent t-test)
- Add: Wilcoxon Signed-Rank Test (non-parametric alternative to paired t-test) — you have Mann-Whitney but this is the missing “paired” counterpart
- Kruskal-Wallis Test (non-parametric alternative to ANOVA)
Stage 5: Categorical data
- Chi-Square Test (goodness of fit, and test of independence)
Stage 6: Correlation
- Pearson Correlation
- Add: Spearman’s Rank Correlation (non-parametric alternative to Pearson) — natural companion, easy win here
Stage 7: Regression
- Linear Regression (simple, then multiple)
- Add: Multicollinearity / VIF when covering multiple regression
- Logistic Regression
Stage 8: Industry-relevant additions (build this as a final module)
These are things practitioners use constantly but academic courses often skip:
- Effect size (Cohen’s d, eta-squared) — a p-value alone doesn’t tell you if a result is practically meaningful, which is exactly what industry cares about
- Multiple testing correction (Bonferroni, Benjamini-Hochberg/FDR) — essential once students run many tests on the same data (common in A/B testing, dashboards)
- Power analysis / sample size calculation — crucial for designing experiments before collecting data
- A/B testing framework — ties together t-test, chi-square, and power analysis into the most common real-world business application
- Bootstrapping / resampling methods — increasingly used in industry when assumptions don’t hold and sample sizes are awkward
——————————————————————————————————–
NEXT STEP (For stage 1 to stage 7)
I used following template for next prompts. I changed the stages (from 1 to 7 only) at each prompt
Prompt 1 to Prompt 7:
You are an expert statistician, data scientist, educator, and Jupyter Notebook author. Your task is to create a complete teaching notebook for a classroom session.
Topic
The notebook should teach the following topic:
<Copy and paste each stage concept here>
Audience
- Undergraduate or postgraduate students with basic Python knowledge.
- Students know NumPy, Pandas, Matplotlib, seaborn, stats, scipy library, but are new to this statistical test.
Duration
Design the notebook for a 40-minute classroom session. The content should be concise enough to comfortably finish within this time.
Dataset
Use an open-source dataset such as:
- seaborn datasets
- scikit-learn datasets
- statsmodels datasets
- scipy datasets
- or create a small synthetic dataset if necessary.
Avoid datasets that require downloading from external websites.
Notebook Structure
Create a well-organized Jupyter Notebook with Markdown explanations and Python code cells.
Include the following sections:
1. Introduction
2. Intuition
3. Theory: Keep mathematical derivations brief and classroom-friendly.
4. Practical Example: Use modern Python libraries where appropriate, such as:
- pandas, numpy, matplotlib, seaborn, scipy.stats, statsmodels
Write clean, well-commented code.
5. Interpretation
Explain every important output
6. Practice Exercise
Include few exercises for students to solve independently.
Do not provide solutions.
NEXT STEP (for stage 8)
Then I provided following prompt for stage 8.
Prompt 8:
Can following topic be covered in one notebook or should I use separate notebook for above. Just answer this question
“””
Stage 8: Industry-relevant additions (build this as a final module) These are things practitioners use constantly but academic courses often skip:
- Effect size (Cohen’s d, eta-squared) — a p-value alone doesn’t tell you if a result is practically meaningful, which is exactly what industry cares about
- Multiple testing correction (Bonferroni, Benjamini-Hochberg/FDR) — essential once students run many tests on the same data (common in A/B testing, dashboards)
- Power analysis / sample size calculation — crucial for designing experiments before collecting data
- A/B testing framework — ties together t-test, chi-square, and power analysis into the most common real-world business application
- Bootstrapping / resampling methods — increasingly used in industry when assumptions don’t hold and sample sizes are awkward
“””
OUTPUT RECEIVED:
One notebook, but it’ll run long — this is denser than Stages 1–7 combined, so plan for it differently:
Recommendation: Split into two notebooks, not one.
Here’s why:
- Each of the 5 sub-topics (effect size, multiple testing correction, power analysis, A/B testing framework, bootstrapping) has enough theory + practical depth to be its own 20-30 minute segment — that’s already 100-150 minutes if done properly, way over your stated 40-minute pattern for other stages.
- A/B testing framework is explicitly described as “tying together t-test, chi-square, and power analysis” — meaning it’s a capstone/integration topic that depends on the other four being taught first. It doesn’t sit well in the same single pass as the foundational pieces.
- Effect size, multiple testing correction, and power analysis are tightly related (all about “is this result real AND does it matter, and did I test correctly”) — natural to group together.
- Bootstrapping is conceptually distinct (resampling-based inference) and pairs better with the A/B testing capstone, since bootstrap CIs are increasingly used in A/B tests.
Suggested split:
| Notebook | Contents | Time |
| 8A: Rigor Toolkit | Effect size (Cohen’s d, eta-squared) + Multiple testing correction (Bonferroni, FDR) + Power analysis | ~40 min |
| 8B: A/B Testing & Bootstrapping | A/B testing framework (integrating t-test/chi-square/power) + Bootstrapping/resampling | ~40 min |
This keeps each notebook at the same digestible length as Stages 1-7, and 8B naturally becomes the “capstone” that shows students how everything (Stages 1-8A) comes together in a real business scenario.
Want me to build it this way?
I replied with following prompt:
Prompt 9:
Yes. Start with 8A: Rigor Toolkit
Prompt 10:
Now do 8B: A/B Testing & Bootstrapping