Who? Where?

futurology is the worst subreddit for factual information.

gary marcus' objections have nothing to do with world models, but the fact that both deep learning and LLM have nothing to do with intelligence the way we see it in biological species i.e. their lack of ability for generalizing. it is fundamentally based on optimizing using gradient learning and this in my view is the opposite route to go when we're trying to engineer general intelligence.

But there's the possibility we could build specialised top class models and in the future we just keep making them more amd more multimodal and general by adding other models on top of it.

Maybe that's another way to AGI. Narrow AI models/agents strewn together such that the sum is greater than the parts.

all my engineering intuition bets against that. but i do get the idea, and i also have a good intuition of what kind of intelligence this kind of approach will give rise to and i'm okay with that. nothing wrong with scaled up LLMs and reinforcement learning. all innovative algorithms are welcome. engineers will keep at it while fancy things distract others.

Some of Marcus' comments are so strange because he always thinks about AGI and seems to to think that other people also think that way. His critique against ChatGPT is that he doesn't think it is a fast way to reach AGI. He basically says we should scrap GPT and do other things. I agree on GPT not being a steep stepping stone towards AGI, I don't think GPT has much at all to do with AGI. But that is not the point! GPT3 is a fantastic tool made to solve lots of things. Even if it never has anything to do with AGI it still worth an insane amount of money and will be extremely beneficial.

For me GPT might be more important than AGI. Each time Marcus speaks he just assumes that everyone's goal is AGI. It is very strange.

I'm sure that Gary Marcus will respond the same way that the average self-proclaimed cynic and skeptic on /singularity does: "no ur wrong and dum lol"

Again?

Yeah, these language models might be one building block, but their output will for example need to be parsed and related to world models and specific knowledge graphs. Also, people have an individual memory and many other elements to them.

if gpt is more important for you that's okay. everyone has a mission and it doesn't have to be the same. there are physicists still going at it without caring much about gpt or agi. who cares man, we have a limited life and we'll all be dead sooner or later. relax.

I am not convinced either or, but it is a strange, and clearly false, assumption that all ML has AGI as a goal. Most of the time people just want to solve a problem.

that's fine and it's a great tool like most tools humans have invented, id even say NN and gradient descent is the greatest idea so far. so what, we must keep going while society makes use of inventions along the way.

just like my roomba!

>gary marcus' objections have nothing to do with world models,

I think they do. See: https://garymarcus.substack.com/p/how-come-gpt-can-seem-so-brilliant . GPT and other LLMs don't are not grounded in the real world, so cannot form an accurate model of them; only getting secondary (from human text). This causes them to make mistakes about relationships; they don't 'master abstract relationships'. I know he doesn't use the term there, but that's what he's getting at.

Also, at https://garymarcus.substack.com/p/how-new-are-yann-lecuns-new-ideas he says:

>A large part of LeCun’s new manifesto is a well-motivated call for incorporating a “configurable predictive world model” into deep learning. I’ve been calling for that for a little while....

The essay isn't primarily about his thoughts on world models, but marcus, for better or worse, thinks that they are important.

disclaimer: idk much about gary marcus, i only follow a few people closely in the field like joscha bach, and i'm sure he wouldn't say or worry about such things.

if you give 3 hands to a generally intelligent neural network, it will figure out how to make use of 3 hands, or no hands. it doesn't matter. so those trivial things are not to be worried about, the problem at hand is different.

[deleted]

>Gary Marcus u/GaryMarcus
>
>Enthusiast: ChatGPT is gonna replace search engines like Google!
>
>Skeptic: Yeah, but it doesn’t really work. sometimes it is amazing, but it often gives you garbage.
>
>Enthusiast: Sure but you can make it work! All you have to do is … hook it up to … a …search engine!
>
>🙄
>
>11:02 PM · Dec 7, 2022

His name was not in the team section on the Robust.AI website, is he really the founder and CEO? He doesn't seem to have much sense as a scientist.