You are also forgetting that there are many many many more scenarios in which AI is catastrophic for humanity rather than beneficial and only a couple scenarios in which it helps more than just the wealthiest who have first access to it.

We have no way of knowing if and when “the singularity” hits (if we even are able to recognize that moment when it happens) that it will be aligned with what is good for humans. The singularity may decide humans would be happiest by not existing!