>ChatGPT is designed to predict the next set of words, or more accurately 'characters' that should come after an input. It does this 100% of the time, and does it's very best at it every single time.

That's not anywhere close to what we could call a 'sentient' AI. And we have already seen that LLMs can develop emergent properties. It does do what it's designed to do, but it also relies on some of the aforementioned emergent properties. Likewise, ChatGPT is static it's incapable of learning from new data in real-time, something that more advanced AI may be capable of doing.

>ChatGPT never attempts to predict something wrong, never refuses to answer a question, unless and excepting if it's programming tells it that it should give those generic stock answers and refuse.

Sure it does. An internal predictor that ChatGPT uses does include false answers, it ranks several answers on the likelihood of the criteria it's designed to follow and goes with the one that best matches that criteria, irrespective of their truthiness or correctness. If you look at the ChatGPT page before you start a conversation there is warning that says that it can provide false or misleading answers. There are situations where ChatGPT answers form a perspective that it cannot have experienced, and it can make faulty logical reasons even for some really basic logic. Sometimes it answers with grammatical errors, and sometimes with garbled nonsense text.

>My side of the field does have evidence, and plenty of it.

If that is the case, then present it in a thesis. Have it pass through the same rigors as all other scientific evidence. If it passes muster, then we may be one step closer to solving this puzzle. Until then it's all speculation.

>I'm taking the historical stance, that AI will continue to act as AI does right now. More advanced AI will get bigger tasks and more complicated solutions, but not fundamentally different until we're past AGI.

This is faulty reasoning, on the basis that the technology and algorithms underlying AI have gone through and will continue to go through revisions, changes, updates, and discoveries. AIs has gone through leaps and bounds from the days of the Paperclip assistant to today's LLMs. Even LLMs have gone through numerous changes that has given them new properties both coded and emergent.

>Really, the biggest question I have, beyond possibilities and theories and unknowns, is why you would assume that things will change in the future, going against historical precedent, to look more like sci-fi?

Historical precedent is change. The AIs of today look nothing like the AIs of 1960s, 2000s, 2010s. And they are displaying new behaviors that are currently being studied. The discussions ranging in the upper echelons of software engineering and mathematics has nothing to do with sci-fi, but observing the newly discovered properties of the underlying algorithms in current gen AI.

>Honestly that's the only source of information that has AI look anything like what people are worried about right now.

Informal discussions on any topic tend to be speculative, that's usually how it goes. Besides, speculating can be fun and depending on the level of the discussion can reveal interesting insights.

>Even for the sake of being prepared and evaluating the future, it just doesn't make sense for so many people, that are pro-AGI no less, to be worried that there's a chance that some level of complexity gives rise to possibility of a great AI betrayal.

People barely trust each other, and discriminate base on looks, it's no surprise that the general population my have concerns regarding something we can barely identify with. And pro-AGI folk are no exception. Likewise, we humans often have concerns with things we don't understand, and the future of things. It's normal.

>I don't know, maybe I'm looking at it wrong, but it really feels like if someone told me that Tesla self driving cars might decide to kill me because the AI in it personally wants me dead. That's the level of absurdity it is for me, I just cannot fathom it.

You should read the short story Sally by Isaac Asimov. Who knows, it could happen one day. Chances are though that if sentient AI can develop its own internal goals, then we probably wouldn't want to put it in a car. But this does bring up a point: even though Teslas are designed not to injure or kill either occupants or pedestrians, sometimes it may still happen given lapses in code or very rare edge cases, it's in these areas that AI goals could manifest.

>In the end, I can say with plenty of evidence, that it is currently impossible for an AI to have internal motivations and goals.

How would you define motivations in software? How would you define goals? Internal goals? Do you have a test to determine either? Do we understand that nature of motivations and goals in biological neural networks? Does your evidence pass the rigors of the scientific method? Are you referencing a body of work, or works, that have?

I do agree that right now we don't seem to observe what we would informally refer to as 'internal goals' in AI, but we're far form saying that it's impossible for it to happen. Just be careful with the use of words in informal and formal contexts and try not to confuse them ('theory' and 'hypothesis' being one such example).

>I can say with evidence and precedent, that in the future AI will change but will be limited to stay as perfectly obedient pieces of software.

We'll see, I guess.