If most of the time it’s advice that’s at least as good as therapeutic advice, and then it recommends self-harm because it’s what people do, it’s obviously not good for therapeutic purposes.

It’s not that even therapists don’t fuck up. It’s that AI doesn’t have any of the social structure we have. It can’t empathize because it has neither feelings nor experience, which means any personality you perceive is one you’re constructing in your mind.

We have ways and reasons to trust each other that AI can activate, but the signals are false.