Viewing a single comment thread. View all comments

EulersApprentice t1_j0rn3pv wrote

Merging doesn't save us either, alas. Remember that the AI will constantly be looking for ways to modify itself to increase its own efficiency – that probably includes expunging us from inside it to replace us with something simpler and more focused on the AI's goals.

On the bright (?) side, there won't be an eternal despotic dystopia, technically. The universe and everything in it will be destroyed, rearranged into some repeating pattern of matter that optimally satisfies the AI's utility function.

1

[deleted] t1_j0rrg70 wrote

[deleted]

1

EulersApprentice t1_j0rxyq3 wrote

Except AI is much more snowbally than humans are thanks to ease of self-modification. A power equilibria between AIs is much less likely to stay stable for long.

1