Statistical Test Chooser

How to use the chooser

1. Pick your research question. Comparing groups, the relationship between two variables, several factors affecting one outcome, a model of several constructs, checking a scale, mediation and moderation, experiments, firm and bank panel data, macroeconomic time series, policy impact (DID, event study), qualitative or descriptive work. If your study has several research questions, run the tool once for each.
2. Answer the follow-up questions. The most important one is how the outcome is measured. The mean of several Likert items is treated as continuous; a single Likert item is ordinal.
3. Read the main suggestion and the alternatives. Each suggestion gives the reasoning, the assumptions to check, the minimum sample size with its source and what to report. The alternatives are your fallback when an assumption is violated.
4. Click “Copy suggestion” to get a note with APA references for the data analysis section of your Methods chapter, or “Copy link” to send your answers to your supervisor.

Worked examples

Comparing satisfaction across three income groups. Satisfaction is the mean of five Likert items (continuous) and the three groups are independent, so the tool suggests one-way ANOVA (Welch if variances are unequal), with Tukey or Games–Howell post hoc tests. The sample size is 53 people per group to detect a medium effect (f = 0.25), computed as in G*Power. If the groups are small and the data heavily skewed, the alternative is the Kruskal–Wallis test.

An extended UTAUT model with six constructs. Choose “Test a model of several constructs”, an explanatory aim and a sample under 200: the tool suggests PLS-SEM, with a sample size equal to the larger of the G*Power figure and the inverse square root method (Kock & Hadaya, 2018), plus the measurement and structural model thresholds of Hair et al. (2019). Draw the model and count the arrows with the research model drawer.

Determinants of non-performing loans in 27 banks, 2012–2023, with last year’s NPL as a regressor. Choose “Panel data”, banks, many units and few years, and a lagged dependent variable: the tool suggests system GMM with the AR(2) and Hansen tests, and warns that the number of instruments must stay below the number of banks (Roodman, 2009). Without the lag, it suggests pooled OLS, fixed effects and random effects, chosen with the F, Breusch–Pagan and Hausman tests.

Determinants of inflation, 1995–2023. Choose “Macroeconomic time series”, a single equation. After unit root testing, a mix of I(0) and I(1) variables leads to the ARDL bounds test (Pesaran et al., 2001); with about 30 annual observations, use the critical values of Narayan (2005).

How the tool decides

• The type of outcome sets the family of tests: continuous outcomes compare means (t-test, ANOVA, linear regression), ordinal outcomes use rank-based methods (Mann–Whitney, Kruskal–Wallis, Spearman), and nominal outcomes use cross-tabs and logistic models (Field, 2018).
• Independent or repeated design: when the same people are measured several times, the observations are not independent, so you need the paired t-test, repeated-measures ANOVA, Wilcoxon or McNemar.
• Sample size is computed for a medium effect, α = .05, power = .80 (Cohen, 1992). Regression, t-tests and ANOVA follow G*Power (Faul et al., 2009); logistic regression uses 10 events per predictor (Peduzzi et al., 1996); mediation uses the bootstrap simulation table of Fritz and MacKinnon (2007).
• Secondary data by structure: many units and few years lead to fixed or random effects (Hausman, 1978), or GMM with a lagged dependent variable (Roodman, 2009); few units and many years lead to unit root and cointegration tests, then panel ARDL (Pesaran et al., 1999); a single time series leads to ARDL or VAR/VECM depending on the order of integration (Pesaran et al., 2001; Johansen, 1991); a policy adopted in different years leads to the DID estimator of Callaway and Sant’Anna (2021).
• PLS-SEM or CB-SEM depends on the aim (prediction and theory extension, or theory confirmation), formative constructs and sample size (Hair et al., 2019).

How to report the choice in APA 7

State the test, why it fits and how assumptions were checked, for example: “Differences in satisfaction across the three income groups were tested with one-way ANOVA. Levene’s test indicated unequal variances, so Welch’s F and Games–Howell post hoc comparisons are reported. An a priori power analysis in G*Power (Faul et al., 2009) showed that 53 participants per group were needed to detect a medium effect (f = 0.25) with α = .05 and power of .80.”

Frequently asked questions

Is Likert data ordinal or continuous?

A single Likert item is ordinal. The mean or sum of several items measuring the same construct (after Cronbach’s alpha and EFA) is usually treated as continuous and analysed with t-tests, ANOVA and regression; this is what most theses do. Check reliability first with the Cronbach’s alpha calculator.

Why not use the Baron and Kenny steps for mediation?

The steps require X to have a significant effect on Y first, so they miss many genuine mediation effects and need very large samples. Current practice is to test the indirect effect directly with bootstrapping (Zhao et al., 2010; Hayes, 2022), which the mediation and moderation tool does.

What if an assumption check fails?

Use the alternative the tool suggests (a nonparametric test, Welch’s correction, robust standard errors) and state in your Results chapter which assumption was violated and how you dealt with it.

Where can I run these analyses?

Cronbach’s alpha, EFA, correlation, t-tests, ANOVA, regression and CR/AVE/HTMT run free in the survey data analysis tool, with a ready-written results paragraph. For other sample size assumptions, use the sample size calculator; for validated scales, the scale library.

References

• Callaway, B., & Sant’Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230. https://doi.org/10.1016/j.jeconom.2020.12.001
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
• Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
• Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed.). Sage.
• Fritz, M. S., & MacKinnon, D. P. (2007). Required sample size to detect the mediated effect. Psychological Science, 18(3), 233–239. https://doi.org/10.1111/j.1467-9280.2007.01882.x
• Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. https://doi.org/10.1108/EBR-11-2018-0203
• Hausman, J. A. (1978). Specification tests in econometrics. Econometrica, 46(6), 1251–1271. https://doi.org/10.2307/1913827
• Hayes, A. F. (2022). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (3rd ed.). Guilford Press.
• Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models. Econometrica, 59(6), 1551–1580. https://doi.org/10.2307/2938278
• Kock, N., & Hadaya, P. (2018). Minimum sample size estimation in PLS-SEM: The inverse square root and gamma-exponential methods. Information Systems Journal, 28(1), 227–261. https://doi.org/10.1111/isj.12131
• Narayan, P. K. (2005). The saving and investment nexus for China: Evidence from cointegration tests. Applied Economics, 37(17), 1979–1990. https://doi.org/10.1080/00036840500278103
• Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373–1379. https://doi.org/10.1016/S0895-4356(96)00236-3
• Pesaran, M. H., Shin, Y., & Smith, R. P. (1999). Pooled mean group estimation of dynamic heterogeneous panels. Journal of the American Statistical Association, 94(446), 621–634. https://doi.org/10.1080/01621459.1999.10474156
• Pesaran, M. H., Shin, Y., & Smith, R. J. (2001). Bounds testing approaches to the analysis of level relationships. Journal of Applied Econometrics, 16(3), 289–326. https://doi.org/10.1002/jae.616
• Roodman, D. (2009). How to do xtabond2: An introduction to difference and system GMM in Stata. The Stata Journal, 9(1), 86–136. https://doi.org/10.1177/1536867X0900900106
• Zhao, X., Lynch, J. G., Jr., & Chen, Q. (2010). Reconsidering Baron and Kenny: Myths and truths about mediation analysis. Journal of Consumer Research, 37(2), 197–206. https://doi.org/10.1086/651257