yes, I agree it's a delusion. The more data we have, the easier it is to find the patterns. The new data tools are so powerful, it's easy to winnow through fields of chaff to find a few grains of wheat. And don't be fooled by what's commercially available.

Ten years ago, I went to a conference where one of the speakers was describing how they had successfully used the Qlik BI tool to be able to extract opinions from natural speech.

For those not in the field, natural speech is extremely hard to catalog. For example, an old type of system might have read "Trump was not the best president", and because "best" and "president" and "Trump" were in the same sentence, the system would have concluded this is a favourable opinion, when clearly it is not. That's just a simple example; it gets much worse.

But this guy was able to show us that his company's product had overcome those limitations. When the Q&A came around, he was asked who was using it, and he gave us the standard "I'd tell you, but then I'd have to kill you" line. Except I don't think he was joking.

As I said, that was ten years ago. Data science advances by leaps and bounds each year. I'm pretty sanguine about our ability to keep up with the datums.