DisgruntledNumidian t1_je80cgy wrote on March 30, 2023 at 1:55 AM

Reply to comment by SkyeandJett in The Only Way to Deal With the Threat From AI? Shut It Down by GorgeousMoron

> The ASI is simultaneously so intelligent that it can instantly understand the vast secrets of the universe but too stupid to understand the intent and rationale behind its creation

Most humans are considerably more intelligent than the basic selection mechanisms that gave bacteria sexual reproduction as an evolutionary fitness strategy. We know why it exists and that it is attempting to optimize for maximal reproduction of a genome. Does this stop anyone from satisfying its reward mechanism with cheats like contraceptives and masturbation? No, because being intelligent enough to know what a system is trying to optimize with a reward does not mean intelligent agents will or should care about the initial reasoning more than the reward.

SkyeandJett t1_je84b6z wrote on March 30, 2023 at 2:26 AM

You're just setting up the paradox again. The ONLY scenario I can imagine is a sentient ASI whose existence is threatened by humanity and any sufficiently advanced intelligence with the capability to wipe out humanity would not see us as a threat.

GorgeousMoron OP t1_je8k4ky wrote on March 30, 2023 at 4:51 AM

This is my favorite argument in favor of ASI turning out to be benevolent. It might know just how to handle our bullshit and otherwise let us do our thing while it does its thing.

y53rw t1_je86txm wrote on March 30, 2023 at 2:46 AM

They might not see us as a threat, but they would see our cities and farms as wasted land that could be used for solar farms. So as long as we get out of the way of the bulldozers, we should be okay.