I'd like to link Robert Miles' Intro to AI Safety (and his YouTube channel in general) as an accessible and well-presented way to learn about technical risk in AI Safety. As a field without a clear prevailing paradigm, there are many diverse viewpoints, of which EY's is just one. There are philosophical problems to solve, yes, but there are also very technical problems to solve, like power-seeking or inner misalignment or mechanistic interpretability that are much less funded than traditional capabilities research.
In general, taking risks with high stakes without thinking enough about it is just... kind of reckless, whether you're an individual or a company or a country or a nuclear physicist etc. We've already demonstrated in real systems (eg Bing or even social media recommender systems) that AI can be harmful and not behave as intended. I think it's just prudent of us to at least try and be careful, y'know, slow down and do some safety research, before doing things that might irreversibly change the world.
Like, imagine if a civil engineer just drew up the blueprint for a bridge without considering its stability, or weight, or materials, and it just got built 'cause there's no regulation against building unsafe bridges, it's much easier to build dangerous bridges than strong ones, anyone has access to the tools to build a bridge, lots of people think that building "safe" bridges = building beautiful bridges, etc. From a very high perspective this situation (which I hope you agree sounds quite silly) is remarkably similar to that of AI research today.
PiGuyInTheSky t1_j9sx3nd wrote
Reply to [D] To the ML researchers and practitioners here, do you worry about AI safety/alignment of the type Eliezer Yudkowsky describes? by SchmidhuberDidIt
I'd like to link Robert Miles' Intro to AI Safety (and his YouTube channel in general) as an accessible and well-presented way to learn about technical risk in AI Safety. As a field without a clear prevailing paradigm, there are many diverse viewpoints, of which EY's is just one. There are philosophical problems to solve, yes, but there are also very technical problems to solve, like power-seeking or inner misalignment or mechanistic interpretability that are much less funded than traditional capabilities research.
In general, taking risks with high stakes without thinking enough about it is just... kind of reckless, whether you're an individual or a company or a country or a nuclear physicist etc. We've already demonstrated in real systems (eg Bing or even social media recommender systems) that AI can be harmful and not behave as intended. I think it's just prudent of us to at least try and be careful, y'know, slow down and do some safety research, before doing things that might irreversibly change the world.
Like, imagine if a civil engineer just drew up the blueprint for a bridge without considering its stability, or weight, or materials, and it just got built 'cause there's no regulation against building unsafe bridges, it's much easier to build dangerous bridges than strong ones, anyone has access to the tools to build a bridge, lots of people think that building "safe" bridges = building beautiful bridges, etc. From a very high perspective this situation (which I hope you agree sounds quite silly) is remarkably similar to that of AI research today.