entanglemententropy

entanglemententropy t1_ixablcw wrote

This ties into a weird idea that I've had for a while, which is that at some point (possibly already), military intelligence might be seriously interested in this question. Meaning that they should have people looking for signs of AGI. Essentially, an AGI could be a very powerful strategic weapon, and indeed we are already seeing some sort of arms race as China, US and the EU is dumping money on AI research. So for security reasons you would want to know if an adversary have developed it, both to try and protect your own interests and also for more offensive purposes like sabotage or technology theft etc. If someone managed to develop AGI without it becoming public knowledge, presumably they would try and use it in non-obvious ways to gain strategic advantage. This sounds like some sci-fi plot, but it might not be too far away.

Along similar lines, I would not be surprised if various militaries eventually will try and have "Manhattan project" style initiatives for developing AI. This would probably be kept under wraps as well, just like the original Manhattan project was; so it could well be that both the US and Chinese military is already spending a lot of money on such things (meaning, a lot more than what we publically know, since clearly DARPA is open about a lot of its AI and robotics research).

2

entanglemententropy t1_ixaa6j8 wrote

I would tend to agree, but it really depends on the capabilities of the AGI and how much more capable it is compared to previous models. If the AGI can do any sort of bootstrap, improving its own capabilities on its own (basically a fast takeoff), then all bets are off. An AGI that achieves superhuman intelligence can surely make money in a lot of ways; not only by playing the markets. There's a whole host of ways of making money online these days, and also remember that such an AGI would probably be capable of generating convincing speech and video, meaning that it could act as a person (or rather, a whole host of persons, both stealing real peoples identities by cloning their voice and face, and also making up fake ones).

However, in the first place it seems highly unlikely that an individual would arrive at AGI before major companies or governments, because of the hardware requirements alone. Not too many individuals can spend millions of dollars on compute for training a large model, which seems to be needed.

1