I think you're on the money. Once we develop more novel network and system structures that are really good at what they do while still generalizing, it will be game over. I think the current models that ML engineers have created are not complex or nuanced enough to extract the kind of value that humans can out of a "small" number of tokens. The human brain is great at having centralized control, coordination across systems, and effective interconnection, and each subsystem can do its "tasks" extremely well and can generalize across tasks too. With that in mind, we are going to need much more complex systems to achieve AGI.