I think we are past that. It would maybe have worked 10 years ago..

My concern is that even the models less powerful than ChatGPT (which can be run on a single pc), can be linked up as components into systems which could achieve AGI. Raw transformer based LLMs may actually be safer than this because they are so alien that they don't even appear to have a single objective function. What they "want" is so context sensitive that they are more like a writhing mass of inconsistent alignments - a pile of masks - this might be really good for us in the short term. They aren't even aligned with themselves. More like raw intelligence.

I also think that approximate alignment will be significantly easier than perfect alignment. We have the tools right now, this approximate alignment is possible. Given the power combined with lack of agency of current LLMs, we may surpass AGI without knowing it. The issue of course is someone just has to set it up to put on the mask of a malevolent or misaligned AI. Thats why I'm worried about concentrating power.

I'll admit I'm out of my depth here, but looking around, so are most of the actual researchers.