Viewing a single comment thread. View all comments

Darth_Face2021 t1_ix3pjuy wrote

Reply to comment by AllanfromWales1 in Honey improves key measures of cardiometabolic health, including blood sugar and cholesterol levels -; especially if the honey is raw and from a single floral source. Honey is a complex composition of common and rare sugars, proteins, organic acids and other bioactive compounds by Wagamaga

What makes it worse is that if you make a lot of claims, it becomes less surprising that one might coincidentally meet some standard for significance.

12

rebuilding-year t1_ix40mli wrote

It even has a name! P-hacking. If you look at 20 different factors, the likelihood that one of them has a statistical significance better than 0.05 is quite high. If you then ignore those that aren't significant and focus only on the one that is, it makes the finding seem more legitimate than it is.

18

usefully_useless t1_ix4gguz wrote

This exactly. Assuming the 20 factors are independent and that the true effect of each factor is zero (i.e. none of them actually do anything), then when using a 5% significance level the probability of finding statistical significance in at least one of the factors (at least one false positive) is about 64%.

There’s a reason that we’re facing a replication crisis, and that reason is the prevalence of p-hacking. (There’s an argument that the overwhelming preference for positive results in academic journals and the publication requirements most departments have for tenure are indirectly responsible for this problem as well, but that’s a different discussion.)

10