> like a mathematical proof that using one intelligence to design another intelligence of equal complexity is an undecidable problem

No, it's not like that. Evolution is not a smart algorithm, but it created us and all life. Even though it is not smart, it is a "search and learn" algorithm. It does massive search, and the result of massive search is us.

AlphaGo wasn't initially smart. It was just a dumb neural net running on a dumb GPU. But after playing millions of games in self-play, it was better than humans. The way it plays is by combining search + learning.

So a simpler algorithm can create a more advanced one, given a massive budget of search and ability to filter and retain the good parts. Brute forcing followed by learning is incredibly powerful. I think this is exactly how we'll get from chatGPT to AGI.