currentscurrents t1_j8zs6o6 wrote on February 18, 2023 at 3:17 AM

Reply to comment by tornado28 in [D] What are the worst ethical considerations of large language models? by BronzeArcher

We will control what the robots want, because we designed them.

That's the core of AI alignment; controlling the AI's goals.

tornado28 t1_j8ztxdg wrote on February 18, 2023 at 3:33 AM

Yeah I guess I'm pretty pessimistic about the possibility of aligned AI. Even if we dedicated more resources to it, it's a very hard problem. We don't know which model is going to end up being the first AGI and if that model isn't aligned then we won't get a second chance. We're not good at getting things right on the first try. We have to iterate. Look how many of Elon Musk's rockets blew up before they started working reliably.

Right now I see more of an AI arms race between the big tech companies than an alignment focused research program. Sure Microsoft wants aligned AI but it's important that they build it before Google, so if it's aligned enough to produce PC text most of the time that might be good enough.

currentscurrents t1_j8zugnd wrote on February 18, 2023 at 3:37 AM

The lucky thing is that neural networks aren't evil by default; they're useless and random by default. If you don't give them a goal they just sit there and emit random garbage.

Lack of controllability is a major obstacle to the usability of language models or image generators, so there's lots of people working on it. In the process, they will learn techniques that we can use to control future superintelligent AI.

tornado28 t1_j8zwrwo wrote on February 18, 2023 at 3:58 AM

It seems to me that the default behavior is going to be to make as much money as possible for whoever trained the model with only the most superficial moral constraints. Are you sure that isn't evil?

currentscurrents t1_j8zy3m4 wrote on February 18, 2023 at 4:10 AM

In the modern economy the best way to make a lot of money is to make a product that a lot of people are willing to pay money for. You can make some money scamming people, but nothing close to the money you'd make by creating the next iphone-level invention.

Also, that's not a problem of AI alignment, that's a problem of human alignment. The same problem applies to the current world or the world a thousand years ago.

But in a sense I do agree; the biggest threat from AI is not that it will go Ultron, but that humans will use it to fight our own petty struggles. Future armies will be run by AI, and weapons of war will be even more terrifying than now.