Yeah, but the “that it wasn’t set” part is the problem. Couldn’t any shmuck ask an open AI to program them a new ai whose sole goal/purpose in life is to violently murder every bunny rabbit on the planet?

I don’t see how we can give people access to an AI capable of building us new software without running into this problem pretty much immediately.

Plus, I imagine every corporation and government will be programming in problematic objectives like “Maximize corporate profit” or “protect America at all costs from all threats foreign and domestic” which will probably result in a ton of ethical issues.