How did we define the “normal” values of blood cholesterol, glucose, pressure, and body temperature…? The answer is statistics. By collecting data from populations, we started seeing where the majority of people stand, in order to define the “normal” values.The latter is a straightforward application of statistics, but when we started comparing groups to each others (i.e. testing a new drug in diseased v.s healthy populations) things started to get complicated. We needed a way to make sure that the data we collected is not due to chance or any other artefact.
Consequently, rules for randomization, data collection, bias …etc were established, and references such as p-values, confidence intervals and others became very crucial in deciding if the hypothesis we are testing is true or not (i.e. the drug is effective or not).
The most famous of the above values in the medical field is the p-value. It tells us if we can accept or throw away the “Null Hypothesis”; which is the hypothesis that there is no difference between the tested groups.
P values evaluate how well the sample data support the devil’s advocate argument that the null hypothesis is true. It measures how compatible your data are with the null hypothesis. How likely is the effect observed in your sample data if the null hypothesis is true? […]
- High p values: your data are likely with a true null.
- Low p values: your data are unlikely with a true null.
On the other hand, looking only on the p-value will definitely lead to a misinterpretation of the statistical data, and here’s where a scientist can mislead their readers.
Science doesn’t lie but scientists well… that’s a different story!
By Lissette Padilla from DNews.
Let’s take this statement:
Drug X lowers blood glucose level, and this result came from a study with a p-value of 0.01 (which is less than 0.05, hence statistically significant).
As a first impression, I can say wow!! that drug is actually a miracle… but this is not enough! In reality if we had such an effective drug we could have cured diabetes once-and-for-all. But again, Looking ONLY on the p-value is NEVER enough in any case… and here’s why;
According to MiniTab three things a p-value can NOT prove:
- It tells us that we have a difference between the groups that are being tested, but it can not reflect the magnitude of difference.
In our case, we won’t know if drug X will lower blood glucose by 2 or by 100 units just by knowing the p-value.
- It tells us we can refute the null hypothesis but in reality this isn’t enough to prove that our alternative hypothesis is true. In simpler terms, you only need one example to prove that a hypothesis is wrong, but alternatively one example is not enough to prove that a hypothesis is true.
Back to our case, we are sure that there is a difference between the tested groups, but we can not prove that the latter is due to drug X just by looking at the p-value.
- Finally, remember that the p-value doesn’t tell us anything about what we are observing… it only tells us the odds of observing it.
So by only knowing the p-value is statistically significant doesn’t prove that taking drug X will definitely regulated your blood sugar.
What we just discussed can cause a misdirection of a whole field of studies… we will be elaborating more in a future post about Evidence Base Medicine and its impact on the medical advancement.
And finally, here is how DNews compiled all these ideas and refreshed our memories about the p-Value in one of their fun and informative videos:
Give Science Bar a Like on Facebook, and never miss an update.