I have read the paper, which is how I know that they scale data and parameters equally, meaning a 10x in data results in a 100x in compute required and hence a 100x in cost.

Assumptions wise Im looking more at the number of words on youtube, your estimate is likely wildly off.

Youre also ignoring that the training time could very well be long enough that it would be a better strategy to wait for better GPUs to come out.

my man you cant just scale cost with number of tokens and not number of parameters

way too many mostly false assumptions in these calculations

I don't think this holds for higher layers of abstraction. If all I need to do is ask to make the porgram faster, and then a model does it for me, that's easy. If I ask it to optimize cache hits and data locality, that task would be more difficult for the prompter (more specific). Depending on the quality of the system, the level of abstraction will eventually reach a point where anyone can code, with very little background knowledge.

Disagree, gave my dad my DALL-E access, and he learned to prompt engineer to reasonable success in an hour. Once you've got quick iteration, specifying what you want becomes trivial.

Not my point, the point was if you've genuinely automated software creation, you've almost immediately automated everything else

As a coder, I feel quite safe. Not because Im denying progress, but because if Im not safe from automation, nobody is. Making me quite safe.

Thats my entire point😅

I feel like pretty soon this is gonna feel like "lucky nature", and not "lucky deepmind", if it doesn't already feel like that

Thank god that, at least AI people, dont care too much about journals, and more about functional and/or elegant ideas

I still cant grasp the reason behind "AI is progressing fast, so singularity will happen this decade". Maybe it will, but without a list of things/milestones needed for the singularity and reasonable estimates for each of them for when we're gonna reach them, (none of which you can completely defend because we don't actually know), such estimates hardly have any degree of certainty.

Its in the author's best interests to show of who they are, misaligning that tends to just result in subtly cheating the system

Peer review in AI has been less and less important though, trial by twitter tends to perform much better

Realistic being a relative term here, Im more on Sam Altman's side who said not too long ago it'll be about 7 years before AI can give a good TED talk completely on its own

Actually we know what the LM is, it's PaLM, developed by google under Jeff Dean.

Anonymous peer review is a fucking joke

I meannn, they had probably written most of the paper by then

Except of course for all the companies that have also been doing this. Saycan (https://arxiv.org/abs/2204.01691) actually manages to perform arbitrary tasks with very decent performance, more transparent research (not in open sourcing, but definitely in explaining their method) and google has much more experience with AI (not that it matters much).

Optimus could become very cool, but Tesla decidedly hasnt got an edge in this space