Suppose you are doing research on jelly beans and their effect on acne.

Suppose also there's actually no effect.

A group of scientists does a study, and finds no effect. Since there's no effect, they don't publish their study.

Around the world, maybe many scientists are doing research on the link (if any) between jelly beans and acne. Maybe it's the color? One group studies purple jelly beans, finds no link, and doesn't publish. Another studies red jelly beans, finds no link, and doesn't publish.

Then one day, just by chance, a group found a relationship that was significant at the 5% level.

This was inevitable, since so many groups of scientists are independently studying the phenomenon, in ignorance of what others are doing.

So now there's a published paper linking green jelly beans to acne.

Even more scientists start doing similar research. What other colours have an effect? Do green jelly worms also "cause" acne?

Since there's a lot of research now, more articles get published - red jelly worms are linked to acne, with a p value of 0.02. Green chiffon cake is linked to acne, with a p-value of 0.03. Nobody publishes the results that show no link.

Eventually, the literature shows a strong relationship between confectionery and acne, especially green, especially with gelatin. Food scientists, dermatologists, regulators rely on this information to provide professional advice and to draft laws. It Science journalists inform the public of this new threat to teen health. Soon "everyone knows" how dangerous green food colouring is...

... But actually no link exists.

If people took the time to replicate the studies, and published the failed replications, this wouldn't happen.

Making the initial paper insist on a stricter level of proof doesn't help, because the whole problem is that negative results aren't being published, and the literature is showing a biased set of results. It would be better to publish the results of every study, so people could see whether that 5% result is something that stands alone, suggesting some real link between two things, or if it's just one of a whole series of similar studies, most of which showed no relationship between the things at all.