I just read a book about this. "The Metamorphosis of Prime Intellect." More fiction than science but still a pretty interesting singularity book if anyone's interested. Kinda fucked up though so be warned lol

Of course GPT-4 is nowhere close to that level yet, but I love the idea that the way to see if an AI system can escape its confines and go rogue is to give it a bunch of money and encourage it to do so.

That's like testing the max weight capacity of a bridge by driving multiple overloaded trucks on it.

That's eerie, like how they think this is worth trying now, that it may be within reach. They aren't like redditors falling for the hype either but experts in their field.

>Preliminary assessments of GPT-4’s abilities, conducted with no task-specific finetuning, found it ineffective at autonomously replicating, acquiring resources, and avoiding being shut down “in the wild.”

>ARC found that the versions of GPT-4 it evaluated were ineffective at the autonomous replication task based on preliminary experiments they conducted. These experiments were conducted on a model without any additional task-specific fine-tuning, and fine-tuning for task-specific behavior could lead to a difference in performance. As a next step, ARC will need to conduct experiments that (a) involve the final version of the deployed model (b) involve ARC doing its own fine-tuning, before a reliable judgement of the risky emergent capabilities of GPT-4-launch can be made

So, don't start collecting canned food yet.