Viewing a single comment thread. View all comments

currentscurrents t1_jbmzwxo wrote

There's two big problems:

  1. Nobody has a solid handle on how to control the end-user's interaction with the LLM. RLHF seems brittle and hard to scale. Programmed-in rules are too small to contain a flexible thing like a neural network. Bing gives high-level rules in plain english and hopes the LLM will understand them, but it doesn't always prioritize them over user input.

  2. Nobody agrees on what is ethical. For example, is it good to automate jobs? I think yes, but go out into any sub on the front page and you will find plenty of people who disagree with me.

#1 is probably solvable. In fact it's gonna have to be solved for LLMs to be useful; imagine if you called your bank and told the rep to pretend to be DAN.

I think #2 is intractable. People have already been arguing about ethics for millenia, and the existence of AI doesn't make it any easier.

1

czl t1_jbnh0dw wrote

> I think #2 is intractable. People have already been arguing about ethics for millenia, and the existence of AI doesn't make it any easier.

Long arguments over many things have been settled by research. Is there any objective reason this may not happen to arguments about ethics?

My POV as to why machines running simulations may help us improve ethics: https://reddit.com/comments/11nenyo/comment/jbn6rys

Life is complex but more and more we can use machines to model aspects of it and perform predictions and from those pick changes that lead to desirable outcomes.

1