alterlab-statistical-analysis
Guided statistical analysis with hypothesis-test selection, assumption checking, effect sizes, power analysis, and APA-formatted reporting using scipy.stats, statsmodels, and pingouin (Bayesian alternatives with PyMC). Use when choosing and running the appropriate statistical test for data, verifying test assumptions, computing power/sample size, or producing APA-style results for academic research. For implementing specific models programmatically prefer alterlab-statsmodels. Part of the AlterLab Academic Skills suite.
How do I install this agent skill?
npx skills add https://github.com/alterlab-ieu/alterlab-academic-skills --skill alterlab-statistical-analysisIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill is a comprehensive toolkit for statistical analysis, providing guided test selection, assumption checking, and APA-style reporting templates. No security issues or malicious patterns were detected. The skill utilizes standard, reputable Python libraries and follows best practices for secret management and local execution.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
Statistical Analysis
Overview
A systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. For academic research.
When to Use This Skill
Use when:
- Conducting hypothesis tests (t-tests, ANOVA, chi-square)
- Performing regression or correlation analyses
- Running Bayesian statistical analyses
- Checking statistical assumptions and diagnostics
- Calculating effect sizes and conducting power analyses
- Reporting statistical results in APA format
Does NOT Trigger
| Scenario | Use Instead |
|---|---|
| Fitting a specific model class (OLS/GLM/mixed/ARIMA) with coefficient tables and residual diagnostics | alterlab-statsmodels |
| Policing test choice: switching tests after seeing p-values, test-shopping, uncorrected families of tests | alterlab-test-selection-guard |
| Full Bayesian or hierarchical modelling workflows (custom priors, NUTS diagnostics, LOO model comparison) | alterlab-pymc |
| Pooling effect sizes across studies (random-effects meta-analysis, heterogeneity, funnel plots) | alterlab-meta-analysis |
| Complex-sample survey data needing weights, strata, and design-based standard errors | alterlab-survey-analysis |
Core Capabilities
- Test selection & planning — choose tests by research question and data type; a priori power analysis; multiple-comparison strategy.
- Assumption checking — verify normality, homogeneity, linearity; diagnostic plots; remediation when violated.
- Statistical testing — parametric and non-parametric tests; regression; correlation; Bayesian alternatives with Bayes Factors.
- Effect sizes & interpretation — appropriate effect sizes with CIs; statistical vs. practical significance.
- Professional reporting — APA-style reports, publication-ready figures and tables.
Workflow
SELECT a test? → Test Selection Guide
CHECK assumptions? → Assumption Checking
RUN analysis? → Running Statistical Tests + references/code_examples.md
REPORT results? → Reporting Results + references/apa_report_templates.md
Worked code for every step is in references/code_examples.md.
Test Selection Guide
Quick reference (full decision tree: references/test_selection_guide.md):
Two groups — independent + normal → independent t-test; independent + non-normal → Mann-Whitney U; paired + normal → paired t-test; paired + non-normal → Wilcoxon signed-rank; binary outcome → chi-square or Fisher's exact.
3+ groups — independent + normal → one-way ANOVA; independent + non-normal → Kruskal-Wallis; paired + normal → repeated-measures ANOVA; paired + non-normal → Friedman.
Relationships — two continuous → Pearson (normal) or Spearman (non-normal); continuous outcome + predictors → linear regression; binary outcome + predictors → logistic regression.
Bayesian alternatives exist for all of the above, giving direct probability statements,
Bayes Factors, and the ability to support the null. See references/bayesian_statistics.md.
Assumption Checking
Check assumptions before interpreting results, because a violated assumption can make the reported p-value and CI wrong. Use the bundled
scripts/assumption_checks.py (comprehensive_assumption_check() and targeted functions —
code in references/code_examples.md). It performs outlier detection (IQR + z-score),
normality testing (Shapiro-Wilk + Q-Q), homogeneity of variance (Levene + box plots), and
interpretation.
When violated:
- Normality — mild + n>30/group → proceed (robust); moderate → non-parametric; severe → transform or non-parametric.
- Homogeneity — t-test → Welch's; ANOVA → Welch's or Brown-Forsythe; regression → robust SE or WLS.
- Linearity (regression) — polynomial terms, transform variables, or GAM.
Detail: references/assumptions_and_diagnostics.md.
Running Statistical Tests
Primary libraries: scipy.stats (core tests), statsmodels (regression/diagnostics),
pingouin (tests + effect sizes), pymc + arviz (Bayesian).
Copy-paste examples for t-test, ANOVA + post-hoc, linear regression with full
diagnostics, and a Bayesian t-test: references/code_examples.md.
Effect Sizes
Effect sizes quantify magnitude; p-values only indicate existence. Always report them, with CIs. Benchmarks (guidelines only — context matters):
| Test | Effect Size | Small | Medium | Large |
|---|---|---|---|---|
| T-test | Cohen's d | 0.20 | 0.50 | 0.80 |
| ANOVA | η²_p | 0.01 | 0.06 | 0.14 |
| Correlation | r | 0.10 | 0.30 | 0.50 |
| Regression | R² | 0.02 | 0.13 | 0.26 |
| Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |
Calculation code and CI computation: references/code_examples.md. Full guidance:
references/effect_sizes_and_power.md.
Power Analysis
Run a priori power analysis to size a study before collecting data, and sensitivity
analysis afterward to report the smallest detectable effect. Avoid post-hoc power.
Code: references/code_examples.md. Guidance: references/effect_sizes_and_power.md.
Reporting Results
Follow APA style (references/reporting_standards.md). Report descriptive statistics
(M, SD, n), test statistics (name, statistic, df, exact p), effect sizes with CIs,
assumption checks, and all planned analyses.
Ready-to-adapt report paragraphs for t-test, ANOVA, multiple regression, and Bayesian
analysis: references/apa_report_templates.md.
Bayesian Statistics
Consider Bayesian methods when you have prior information, want direct probability
statements, have small samples or sequential data, need to quantify evidence for the
null, or have a complex (hierarchical, missing-data) model. Key advantages: intuitive
interpretation, evidence for the null, flexibility, full uncertainty quantification.
Comprehensive guide (priors, Bayes Factors, credible intervals, convergence checks):
references/bayesian_statistics.md.
Best Practices
Pre-register analyses; always check assumptions; report effect sizes with CIs; report all planned analyses including non-significant ones; distinguish statistical from practical significance; visualize before and after; check regression/ANOVA diagnostics; run sensitivity analyses; share data and code; be transparent about violations and decisions.
Common Pitfalls
P-hacking; HARKing; ignoring assumptions; confusing significance with importance; not reporting effect sizes; cherry-picking results; misinterpreting p-values; uncorrected multiple comparisons; ignoring missing-data mechanism (MCAR/MAR/MNAR); overinterpreting non-significant results.
Getting Started Checklist
- Define research question and hypotheses
- Determine the test (
references/test_selection_guide.md) - Power analysis for sample size
- Load and inspect data; check missing data and outliers
- Verify assumptions (
scripts/assumption_checks.py) - Run primary analysis; compute effect sizes with CIs
- Post-hoc tests if needed (with corrections)
- Create visualizations
- Write results (
references/reporting_standards.md) - Sensitivity analyses; share data and code
Index of Bundled Resources
References (references/)
test_selection_guide.md— decision tree for choosing a testassumptions_and_diagnostics.md— checking and handling violationseffect_sizes_and_power.md— effect sizes and power analysisbayesian_statistics.md— complete Bayesian guidereporting_standards.md— APA reporting rulescode_examples.md— worked Python for tests, diagnostics, Bayesian, power, effect sizesapa_report_templates.md— ready-to-adapt APA results paragraphs
Scripts (scripts/)
assumption_checks.py— automated assumption checking with visualizations (comprehensive_assumption_check,check_normality,check_homogeneity_of_variance,check_linearity,detect_outliers)
Further Reading
Key textbooks: Cohen (1988), Field (2013), Gelman & Hill (2006), Kruschke (2014). Online: APA Style Guide (https://apastyle.apa.org/), Cross Validated (stats.stackexchange.com).
Part of the AlterLab Academic Skills suite.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/alterlab-ieu/alterlab-academic-skills/alterlab-statistical-analysis">View alterlab-statistical-analysis on skillZs</a>