Yeah, agreed it is probably years out. Just saying…Jesus. This is gonna be fucked up!

We don’t know what it’s goals are. We have a rough idea of the goals it’s been given by engineers attempting to output stuff that will please humans. We don’t know how it could interpret these goals in a way that might be unintended.

Ok but with it they’re now vulnerable to nonstop catfish scams and manipulation by a generative model that seems to be hard to control. That’s obviously a little scarier than the worst case scenario being having a lot of cats

It doesn’t have to be sentient to be terrifying. People’s brains have been broken just by 15 years of a photo sharing app. People are going to fall in love with this thing. People may be manipulated by it, not because it has humanoid goals or motivations but because people are fragile and stupid. It’s barely been available and it’s already obvious that the engineers who built it can’t really control it.

AI doesn’t have to be sentient to cause massive social problems.

I’m not underestimating future iterations but you’re totally missing my point - accuracy is not solved by more data. It is solved by better modeling.

Of course I think the tech will improve, I just think accuracy is not solved by more training data

Not really. One of my colleagues asked ChatGPT for a list of celebrities who shared a birthday with him. The list was wrong - ChatGPT had hallucinated false birthdays for a number of celebrities.

Brad Pitt’s birthday is already in ChatGPT’s training data. More or better training data can’t fix this problem. The issue is that it is outputting false information because it is designed to output words probabilistically without regard for truth. Hallucinations can only be solved manually be reinforcing good responses over bad responses but even if it gets better at outputting good responses, it still will have an issue with creating hallucinations in response to novel prompts. Scale isn’t a panacea.