15th February 2024

p-value

Statistical tests are used to test the null hypothesis. This is done by calculating the p-value to determine statistical significance.

p-value is the probability of getting the observed values you obtained in your dataset or something more extreme, provided that the null hypothesis is true. To put things in another way, let us look at an example:

You are examining the difference in patient outcomes between drug A and standard care.

Null hypothesis: There is no difference in patient outcomes between patients given drug A and standard care.

Your observed values showed the following:

Patients given drug A have a survival of 90%

Patients on standard care have a survival of 10%

This in itself would be reasonable to think that your findings are significant. However, how can you be sure? The result could be obtained by pure chance with small sample sizes or sampling bias, etc. This is where statistical tests and p-values come in.

Statistical tests were performed, and the p-value turned out to be 0.03.

This means:

If the null hypothesis is true: In other words, in reality, there is no actual difference between patient survival between both groups, and the probability of you obtaining the results above is 0.03 (3%), which is a very low probability. The differences in survival in your results are likely a true finding (a true positive).

If the p-value is 0.5: This means that the probability of you obtaining the results above if the null hypothesis is true is 50%. This is a very high probability. In other words, this finding is likely untrue (a false positive).

By convention, the p-value of <0.05 is chosen to mean statistical significance. However, there is no golden ‘cut-off’ for p-value; the choice of what cut-off for p-value is statistically significant depends on multiple factors – the area being researched, the sample size, the differences being detected, etc.

A statistically significant finding may not be clinically significant. To demonstrate the point, let us look at another example:

You are doing a study examining the difference in patient outcomes between intervention B and standard care.

Null hypothesis: There is no difference in patient outcomes between patients given drug A and standard care.

Your observed values showed the following:

Patients receiving intervention B have a survival of 10.2%

Patients on standard care have a survival of 10%

Statistical tests performed gave a p-value of 0.0001.

With a p-value of <0.05, the findings are statistically significant.

However, is the survival difference of 0.2% clinically significant? This is another question to consider when reviewing medical literature.