> The ASI is simultaneously so intelligent that it can instantly understand the vast secrets of the universe but too stupid to understand the intent and rationale behind its creation
Most humans are considerably more intelligent than the basic selection mechanisms that gave bacteria sexual reproduction as an evolutionary fitness strategy. We know why it exists and that it is attempting to optimize for maximal reproduction of a genome. Does this stop anyone from satisfying its reward mechanism with cheats like contraceptives and masturbation? No, because being intelligent enough to know what a system is trying to optimize with a reward does not mean intelligent agents will or should care about the initial reasoning more than the reward.
DisgruntledNumidian t1_je80cgy wrote
Reply to comment by SkyeandJett in The Only Way to Deal With the Threat From AI? Shut It Down by GorgeousMoron
> The ASI is simultaneously so intelligent that it can instantly understand the vast secrets of the universe but too stupid to understand the intent and rationale behind its creation
Most humans are considerably more intelligent than the basic selection mechanisms that gave bacteria sexual reproduction as an evolutionary fitness strategy. We know why it exists and that it is attempting to optimize for maximal reproduction of a genome. Does this stop anyone from satisfying its reward mechanism with cheats like contraceptives and masturbation? No, because being intelligent enough to know what a system is trying to optimize with a reward does not mean intelligent agents will or should care about the initial reasoning more than the reward.