I think they give GPT-4 a task, GPT-4 attempts to complete it and is told if it worked or not, then GPT-4 looks at what happened and determines why it failed, and then tries again with this new knowledge. This is all done through natural language prompts, the model isn't being changed.

I saw somebody else in either this sub or /r/openai using a very similar method to get GPT-4 to write and deploy a webpage that could accept valid email addresses. Of course, I can't find it, and neither can Bing Chat, so maybe I dreamed it. I distinctly remember asking if it could do QA, and then the person asked what I meant, and I said have it check for bugs. I post a lot so I can't find it in my post history.

I remember the way it worked was they gave it the task, then GPT-4 would write out what it was going to do, what it predicted would happen, write the code, and then check if what it did worked. If it didn't work it would write out why it didn't work, plan again, then act again. So it went plan->predict->act->check->plan. This successfully worked as it went from nothing to a working and deployed webpage without any human intervention other than setting the task.

me too