15th February 2024

Parametric vs. non-parametric tests

Statistical tests can also be subcategorised into parametric and non-parametric tests.

In parametric tests, the sample data distribution is known and based on fixed parameters. For example, the Student’s t-test for two independent samples is only reliable if both samples have a normal distribution with homogenous variance. A parametric test has more statistical power when compared to its non-parametric equivalent.

Non-parametric tests do not assume any distribution of sample data. Non-parametric tests are more robust than parametric equivalent because non-parametric tests are valid in a broader range of situations.

Statistical tests for continuous data

Unpaired student T-test or Independent T-test

• Parametric test
• Compares the means of two independent groups of values

Paired Student t-test

• Parametric test
• Compares the means of two correlated groups, e.g. studies where the same people are measured at two different time points

Analysis of variance (ANOVA) test

• Parametric test
• Compares the means of more than two groups of values
• Better test when compared to performing multiple unpaired Student t-tests. Type 1 error (probability of false positive) increases when multiple unpaired Student t-tests are performed on the same dataset.

Mann Whitney U-test

• Also known as Wilcoxon rank-sum test
• Non-parametric equivalent to unpaired Student t-test

Wilcoxon sign-rank test

• Also referred to as the Wilcoxon matched pairs test
• Non-parametric equivalent to paired Student t-test

Kruskal-Wallis test

• Non-parametric equivalent to ANOVA

Statistical tests for categorical data

Chi-squared test

• Non-parametric test
• Compares proportions of two or more groups

McNemar’s Chi-squared test

• Non-parametric test
• Equivalent of Chi-squared test for paired data

Fisher’s exact test (aka Fisher’s exact probability test)

• Non-parametric test
• The equivalent of the Chi-squared test for sparse data.