Think of it more like this: the total intelligence/capability of all humans greatly exceeds the total intelligence/capability of any individual human. When we train ai models, we are imbuing them with capabilities derived from many humans. But once produced, they are easily copied so we could quickly have a population of them, where each individual ai is equivalent to many humans (and thus greater than any individual human) but maybe less capable than all humans. Eventually the population of enough such ais is more capable than the population of humans.

And then, if the ais can train new generations from prior ais, this pattern could repeat and explode.