I agree, its not the harm from AI itself directly toward humans, its the harm caused by humans who have access to AI as a tool and wield it to influence society to their world view. We are already in a position where a select few decide the rules of the game we play call society, and the select few usually win at this game. Handing over a game genie full of cheat codes to the select few that generally always win, further solidifies their position of power to stamp out any unwanted change by that group, regardless the impact it has to the rest of the world, so long as the select few are nice and comfy in their ivory towers. If you believe for a second that those with the ability to harness powerful AI for their own personal benefit would be benevolent enough to share said tool with the rest of us, I have beach front property in a land locked region to sell you for a very affordable price.