Viewing a single comment thread. View all comments

KingRamesesII t1_jarsmn5 wrote

My original comment was really analyzing an unlikely scenario of aligned narrow AI, or severely limited AGI with proper controls put in place to keep it at roughly human intelligence. This was in order to “play along” with the economic implications of enough robots for everybody to have their own robot. It would be a miracle if we end up here.

I’m not sure “countries” will be a thing after actual AGI.

Another facet of this is that the first country/organization to develop AGI rules the planet, if they can even align the thing. AGI is ASI because narrow AI is already superhuman in every narrow case.

AGI will fight wars, create super weapons, and make current super weapons obsolete, and it will be able to simulate thousands of years of human level research/effort in mere minutes or hours. And such a thing will almost definitely not be controlled by humans.

As Sam Harris says, sure it’s easy to outsmart your teenager. But if your teenager has 20,000 years to respond to your every move, you’re not going to outsmart your teenager. Now imagine what’s possible if that teenager is smarter than every human that has ever lived, combined.

3