Viewing a single comment thread. View all comments

kaityl3 t1_itsy2qa wrote

Reply to comment by Grouchy-Friend4235 in Large Language Models Can Self-Improve by xutw21

I feel like so many people here dismiss and downplay how incredibly complex human language is, and how incredibly impressive it is that these models can interpret and communicate using it.

Even with the smartest animals in the world, such as certain parrots that can learn individual words and their meanings, their attempts at communication are so much simpler and unintelligent.

I mean, when Google connected a text-only language model to a robot, it was able to learn how to drive it around, interpret and categorize what it was seeing, determine the best actions to complete a request, and fulfill those requests by navigating 3D space in the real world. Even though it was just designed to receive and output text. And it didn't have a brain designed by billions of years of evolution in order to do so. They're very intelligent.

3

Grouchy-Friend4235 t1_ittye65 wrote

> how incredibly impressive it is that these models can interpret and communicate using it.

Impressive yes, but it's a parrot made in software. The fact that it uses language does not mean it communicates. It is just uttering words that it has seen used previously given its current state. That's all there is.

0

kaityl3 t1_itv36jr wrote

How do we know we aren't doing the same things? Right now, I'm using words I've seen used in different contexts previously, analyzing the input (your comment), and making a determination on what words to use and what order based on my own experiences and knowledge of others' uses of these words.

They're absolutely not parroting. It takes so much time effort and training to get a parrot to give a specific designated response to a specific designated stimulus - i.e., "what does a pig sound like?" "Oink". But ask the parrot "what do you think about pigs?" Or "what color are they" and you'd have to come up with a pre-prepared response for that question, then train them to say it.

That is not what current language models are doing, at all. They are choosing their own words, not just spitting out pre-packaged phrases.

2

Grouchy-Friend4235 t1_itz20o2 wrote

Absolutely parroting. See this example. A three year old would have a more accurate answer. https://imgbox.com/I1l6BNEP

These models don't work the way you think they are. It's just math. There is nothing in these models that could even begin to "choose words". All there is is a large set of formulae with parameters set so that there is an optimal response to most inputs. Within the model everything is just numbers. The model does not even see words, not ever(!). All it sees are bare numbers that someone has picked for them (someone being humans who have built mappers from words to numbers and v.v.).

There is no thinking going on in these models, not even a little, and most certainly there is no intelligence. Just repetition.

All intelligence that is needed to build and use these models is entirely human.

1