Purplekeyboard

Purplekeyboard t1_je8m61n wrote

Reply to comment by currentscurrents in [R] The Debate Over Understanding in AI’s Large Language Models by currentscurrents

> LLMs likely have a type of understanding, and humans have a different type of understanding.

Yes, this is more of a philosophy debate than anything else, hinging on the definition of the word "understanding". LLMs clearly have a type of understanding, but as they aren't conscious it is a different type than ours. Much as a chess program has a functional understanding of chess, but isn't aware and doesn't know that it is playing chess.

13

Purplekeyboard t1_je8l78y wrote

Reply to comment by currentscurrents in [R] The Debate Over Understanding in AI’s Large Language Models by currentscurrents

The point is that GPT-3 and GPT-4 can synthesize information to produce new information.

One question I like to ask large language models is "If there is a great white shark in my basement, is it safe for me to be upstairs?" This is a question no one has ever asked before, and answering the question requires more than just memorization.

Google Bard answered rather poorly, and said that I should get out of the house or attempt to hide in a closet. It seemed to be under the impression that the house was full of water and that the shark could swim through it.

GPT-3, at least the form of it I used when I asked it, said that I was safe because sharks can't climb stairs. Bing Chat, using GPT-4, was concerned that the shark could burst through the floorboards at me, because great white sharks can weigh as much as 5000 pounds. But all of these models are forced to put together various bits of information on sharks and houses in order to try to answer this entirely novel question.

9

Purplekeyboard t1_jcc7cuo wrote

Reply to comment by ScientiaEtVeritas in [D] What do people think about OpenAI not releasing its research but benefiting from others’ research? Should google meta enforce its patents against them? by [deleted]

But without OpenAI, who would have spent the billions of dollars they have burned through creating and then actually giving people access to models like GPT-3 and now GPT-4?

You can use GPT-3, and even versions of GPT-4, today. Or you can stand and look up at the fortress of solitude that is Google's secret mountain lair where models are created and then hoarded forever.

0

Purplekeyboard t1_j9xz3ir wrote

Reply to The Job Market Apocalypse: We Must Democratize AI Now! by Otarih

"Democratizing" image generation, if that means giving people access to it free, would not be difficult. Imagegen is not that expensive. You can buy unlimited AI image generation now for $25/month from NovelAI (although they only have anime models, but photorealistic models are not more expensive to run).

This also comes with unlimited text generation, although using smaller, weaker models than the best ones available. ChatGPT is currently free as well, and it is the best text generation model that's been released as of yet.

So, at least as long as you live in a first world country, these types of AI are easy to get access to.

2

Purplekeyboard t1_j9ggpzb wrote

Reply to [OC] I asked Georgians (U.S.) if they learned in school about the 1912 racial cleansing in Forsyth County (GA), only 11% of respondents were taught this. by JPAnalyst

Keep in mind that a substantial number of people couldn't even tell you what century World War II was fought in, or what countries were on which sides in that war.

I find it to be unlikely that adults decades later would have any idea if they were taught in school about a 1912 Forsyth County incident. Self reports are not the way to know what is or was taught in school.

10

Purplekeyboard t1_j1j5pdk wrote

Reply to ELI5 What is the underlying principle that lets the creators of ChatGPT (for example) feel confident that it will accurately provide answers to questions they themselves haven’t pondered? by onlyouwillgethis

ChatGPT is essentially just a text predictor.

It is trained on basically all the text on the internet, and it uses this to learn what words tend to follow what other words. It's very powerful and sophisticated, to the point where it can write proper english sentences which are on topic and which are (mostly) accurate.

So if you say, "Where was Elvis Presley born?", it predicts that after this text would generally come text which gives the answer to the question, and that's the text it gives you. And because it has been trained on the text of the entire internet, it knows the answer to this question.

If you say, "Please write me a brief essay on the difference between capitalism and socialism", it predicts how such an essay would likely start, then writes that text. Then predicts how such an essay would likely continue, then writes that text. And so on, until the essay is completed. As it's been trained on the text of the internet, it has large volumes of text in its training material about capitalism and socialism and the differences between them.

ChatGPT is specifically trained to be a chat bot, and it probably has multiple censorship routines and "be a nice chatbot" routines which identify when your prompt or its own writing is something against its rules.

2

Purplekeyboard t1_j12uk1s wrote

Reply to comment by master3243 in [R] Nonparametric Masked Language Modeling - MetaAi 2022 - NPM - 500x fewer parameters than GPT-3 while outperforming it on zero-shot tasks by Singularian2501

But what I'm asking is, how do the benchmarks match real world performance? Because I've seen claims that other language models were supposedly close to or equal to GPT-3 in this or that benchmark, but try interacting with them and the difference is striking. It's like the difference between talking to a college grad student and talking to the meth addled homeless guy who shouts at lampposts.

12