> If ChatGPT can generate code from simple prompts, then what's stopping OpenAI from setting up a positive coding feedback loop for it to work on its own fork of itself? > > I'll come right out and say it: why isn't ChatGPT the seed for a proto-AGI?

Being generous, the code written by ChatGPT is at best at the level of a mediocre first year IT student. It can write simple boilerplate based on solutions it's already seen, but has limited ability to actually solve complex problems.

This is still an incredibly impressive achievement and it blows my mind every time I see it in action. But it's about as likely to make the next major breakthrough in AI research as our imaginary mediocre first year IT student is.

It's hard not to imagine a point where AI is able to improve itself faster than humans can, thus essentially writing the next version of itself. But we're not there yet.

The code isn't good enough nor does the AI have a good enough understanding to implement the code itself. It's all quite basic code and often has errors.

The OpenAI team might be using ChatGPT to help design the next iterations but it's not exactly possible to "cut out the middleman" at the moment All the games/programs that have been made using ChatGPT have only been possible because people can actually check if the code is good, see if the program works and write followup prompts if necessary. It's not within ChatGPT's capabilities to actually see if it's code works. It may spit out a working answer sometimes but it can't actually test if it works itself and as such it can't improve without the aid of a human.

Complex understanding and reasoning of the world is necessary to do so. ChatGPT is still just going to spit out what it feels will be the most probable code and not something insightful. Well, it can actually, but that's an emergent property that is weak at best. With big enough dataset and parameters it could identify relationships and become complex enough to do recursive self improvement but it would take a lot of money to do that.

Somebody posted a while ago how LLMs are animal intelligence like us and not true intelligence. That seems true, we haven't truly cracked intelligence. The thing is we may not need to. We will eventually be able to train bigger, better and more complex models and get to human level intelligence.

Is there a better GPT for code?

>It can write simple boilerplate based on solutions it's already seen,

It can solve Advent of Code problems...

Mediocre first year IT students can do that. But no way it's writing an efficient flash attention kernel without having seen one before.

its memory isn't long enough to write a full program, let along a full model, without at least some help. And its not able to create new concepts and make discoveries, it can only build what has its been trained on.

We still have some breakthroughs needed before AGI.

Also, chatGPT doesn't generate the full code because memory limits. people that made things using chatgpt had to go function by function to get the full program. you can't just go "generate flappy bird". it's more like "generate a bird pmg" then "generate a flapping animation" "generate the obstacles" etc

Yup, exactly. Whilst ChatGPTs ability to code is impressive it’s definitely not the “do everything” button some people think it is.

You are overestimating the abilities of mediocre IT students

Even if it could code better than humans (like AlphaCode, that outperforms most humans in coding competitions), that's not the hard part.

The hard part is the science/engineering aspect of machine learning, programming is just the implementation of the ideas when they are already thought out. Actually coming up with useful improvements is significantly harder and requires a thorough grasp of the mathematical underpinnings of ML. ChatGPT is nowhere near capable of making useful contributions to the machine learning research community (or in other words, capable of writing a ML paper), and therefore it is incapable of improving its own software. AI most likely will reach that level at some point however, possibly in the near future.

A software developer needs to understand the functional designs, technical designs, architecture, and test plans that are relevant. Being able to produce functions is not enough. In a machine learning context knowledge of the concept and mathematics are required. ChatGPT is a level higher than a Markov chain. It has a sense of which words and groups of words go together. But for it words are just lists of numbers. So in fact it evaluates nested functions with vectors as input. For a much smaller network, you can do the same thing in excel.

>It can solve Advent of Code problems...

Which is a collection of relatively simple problems, commonly solved by first year students, where the solutions are almost certainly in the training data set....

A monkey using a typewriter could eventually recreate the complete works of Shakespeare just by pressing random keys if you let him go for long enough.

Even if you devise a machine that could reprogram itself with some additional functionality, however minute, and then run a diagnostic to determine if there was some increase in capability and repeat that process forever until the intelligence parameter is maximized, it might take millions of years and a shit ton of computing resources to get anywhere meaningful with todays AI technology. For that approach to work we would need to first meet a certain threshold of general intelligence to make meaningful enough self-improvements to lead anywhere interesting in a timely manner.

I am one😭

advent of code 2022 was ongoing when people were using ChatGPT for solving problems (although not all of them) it couldn't have been in the data set.

ChatGPT can't code better than a first year but DeepMind's AlphaCode can certainly code better than ever most median quality developers.

So let's rephrase the question to focus on AlphaCode instead of ChatGPT.

How does that change your response, if at all?

What's being ignored in your idea is that ChatGPT itself doesn't consist of code. It's a large artificial neural network, trained on huge amounts of data. It's not an algorithm. The solution of the problem of making it better is not to throw more coding power onto it. Instead you need better concepts, more/better training data, huge amounts of computing time, more human feedback. You can't just take a "chunk of code" of the network and "improve it".

This people

> DeepMind's AlphaCode can certainly code better than ever most median quality developers.

AlphaCode does well at solving quiz questions. From my own experience with those types of quizzes, they're mostly just maths questions solved with code.

Doing well at those types of questions has very little bearing on most real world software engineering.

Now to be fair, machine learning is a lot more math focused than typical software engineering. But if we're going with the assertion that "AlphaCode can certainly code better than ever most median quality developers" based on doing well at quiz questions, then I'm going to disagree.

> So let's rephrase the question to focus on AlphaCode instead of ChatGPT. > How does that change your response, if at all?

Not Really.

Don't get me wrong, AlphaCode is still mind blowing. I really don't want to understate how impressive it is. But I don't think it's at the level of being able to implement itself. Yet.

(Disclaimer: I am not an AI researcher, so take my opinion with a grain of salt).

The specific questions may not have been in the data set, but it'll have seen the same types of questions before.