ajt9000 t1_ivics5n wrote on November 8, 2022 at 4:19 AM

Reply to comment by shumpitostick in The big data delusion – the more data we have, the harder it is to find meaningful patterns in the world. by IAI_Admin

The main way its gonna make a statistical model worse is by increasing the computational power needed to run it. Thats not an argument about the quality of the model results though. I agree the author's understanding of statistics is really bad.

shumpitostick t1_ivlb6zi wrote on November 8, 2022 at 8:16 PM

I was oversimplifying my comments a bit. There is the curse of dimensionality. And in causal inference if you just use every variable as a confounder your model can also get worse because you're blocking forward paths. But if you know what you're doing it shouldn't be a problem. And I haven't met any ML practitioner or statistician who doesn't realize the importance of getting to understand your data and making proper modelling decisions.