I think the big problem when either statisticians or software developers make errors like this is that even if a researcher discovers a discrepancy... it is assumed the researcher did something wrong. And then, they end up on some wild goose chase scratching their heads. If SPSS is really calculating the rank test incorrectly... OMG! Not sure that's "researcher degrees of freedom" but maybe the authors are being sarcastic themselves.
I have tried to run these kinds of discrepancies down. If you aren't trying to be a statistician, it's easier to just write it up in a paragraph how you solved it, and life goes on. No one would ever really put two and two together though. Maybe these authors were willing to take that further.
topgallantswain t1_j0sx1x2 wrote
Reply to A study finds that researcher degrees of freedom in statistical software contribute to unreliable results. Specifically, multiple inconsistencies were found in the results produced between statistical packages due to algorithmic variation, computational error, and statistical output. by Alysdexic
I think the big problem when either statisticians or software developers make errors like this is that even if a researcher discovers a discrepancy... it is assumed the researcher did something wrong. And then, they end up on some wild goose chase scratching their heads. If SPSS is really calculating the rank test incorrectly... OMG! Not sure that's "researcher degrees of freedom" but maybe the authors are being sarcastic themselves.
I have tried to run these kinds of discrepancies down. If you aren't trying to be a statistician, it's easier to just write it up in a paragraph how you solved it, and life goes on. No one would ever really put two and two together though. Maybe these authors were willing to take that further.