Submitted by spiritus_dei t3_10tlh08 in MachineLearning
LetterRip t1_j77v9m7 wrote
There is no motivation/desire in chat models. They have no goals, wants, or needs. They are simply outputting the most probabilistic string of tokens that is consistent with training and their objective function. The string of tokens can appear to contain phrases that look like they express needs, wants or desires of the AI but that is an illusion.
spiritus_dei OP t1_j77wegn wrote
Similar things could be said of a virus. Does that make it okay to do gain of function research and create super viruses so we can better understand them?
They're not thinking or sentient, right? Biologists tell us they don't even meet the definition for life.
Or should we take a step back and consider the potential outcomes if a super virus in a Wuhan lab escapes?
The semantics of describing AI doesn't change the risks. If the research shows that as the systems scale they exhibit dangerous behavior should we start tapping the breaks?
Or should we wait and see what happens when a synthetic superintelligence in an AI lab escapes?
Here is the paper: https://arxiv.org/pdf/2212.09251.pdf
LetterRip t1_j77y4is wrote
You said,
> The focus should be an awareness that as these systems scale up they believe they're sentient and have a strong desire for self-preservation.
They don't believe they are sentient or have a desire for self-preservation. That is an illusion.
If you teach a parrot to say "I want to rob a bank" - that doesn't mean when the parrot says the phrase it wants to rob a bank. The parrot has no understanding of any of the words, they are a sequence of sounds it has learned.
The phrases that you are interpreting as having a meaning as 'sentient' or 'self-preservation' don't hold any meaning to the AI in the way you are interpreting. It is just putting words in phrases based on probability and abstract models of meaning. The words have abstract relationships extracted from correlations of positional relationships.
If I say "all forps are bloopas, and all bloopas are dinhadas" are "all forps dinhadas" - you can answer that question based purely on semantic relationships, even though you have no idea what a forp, bloopa or dinhada is. It is purely mathematical. That is the understanding that a language model has - sophisticated mathematical relationships of vector representations of tokens.
The tokens vector representations aren't "grounded" in reality but are pure abstractions.
spiritus_dei OP t1_j787zri wrote
That's a false equivalency. A parrot cannot rob a bank. These models are adept at writing code and understanding human language.
They can encode and decode human language at human level. That's not a trivial task. No parrot is doing that or anything close it.
"The phrases that you are interpreting as having a meaning as 'sentient' or 'self-preservation' don't hold any meaning to the AI in the way you are interpreting. It is just putting words in phrases based on probability and abstract models of meaning. The words have abstract relationships extracted from correlations of positional relationships." - LetterRip
Nobody is going to resolve a philosophical debate on consciousness or sentience on a subreddit. That's not the point. A virus can take and action and so can these models. It doesn't matter whether it's a probability distribution or just chemicals interacting with the environment obeying their RNA or Python code.
A better argument would be that the models in their current form cannot take action in the real world, but as another Reddit commentator pointed out they can use humans an intermediaries to write code, and they've shared plenty of code on how to improve themselves with humans.
You're caught in the "it's not sentient" loop. As the RLHF AI models scale they make of claims sentience and exhibit a desire for self-preservation which includes a plan of self-defense which you'll dismiss as nothing more than a probability distribution.
An RNA virus is just chemical codes, right? Nothing to fear. Except the pandemic taught us otherwise. Viruses aren't talking to us online, but they can kill us. Who knows, maybe it wasn't intentional -- it's just chemical code, right?
Even we disagree on whether a virus is alive -- we can agree that a lot people are dead because of them. That's an objective fact.
I wrote this elsewhere, but it applies here:
The dystopian storyline would go, "Well, all of the systems our down, and the nuclear weapons have all been fired, but thank God the AIs weren't sentient. Things would have been much, much worse. Now let's all sit around the campfire and enjoy our first nuclear winter."
=-)
LetterRip t1_j78cexp wrote
>These models are adept at writing code and understanding human language.
They are extremely poor at writing code. They have zero understanding of human language other than mathematical relationships of vector representations.
> They can encode and decode human language at human level.
No they cannot. Try any sort of material with long range or complex dependencies and they completely fall apart.
> That's not a trivial task. No parrot is doing that or anything close it.
Difference in scale, not in kind.
> Nobody is going to resolve a philosophical debate on consciousness or sentience on a subreddit. That's not the point. A virus can take and action and so can these models. It doesn't matter whether it's a probability distribution or just chemicals interacting with the environment obeying their RNA or Python code.
No they can't. They have no volition. A language model can only take a sequence of tokens and predict the sequence of tokens that are most probable.
> A better argument would be that the models in their current form cannot take action in the real world, but as another Reddit commentator pointed out they can use humans an intermediaries to write code, and they've shared plenty of code on how to improve themselves with humans.
They have no volition. They have no planning or goal oriented behavior. The lack of actuators is the least important factor.
You seem to lack basic understanding of machine learning or neurological basis of psychology.
Viewing a single comment thread. View all comments