More to do with interest rates, copying Musk, and some overhiring during covid.

Schmidhuber prior art

Deepmind put out a paper on adjusting the pruning mask during training (by reviving pruned weights if a transiently stored gradient exceeds some threshold).

The paper is called Rigging the Lottery (referencing initial weight lottery hypothesis) and method RigL I think.

Didn't they just release Whisper for everyone, including the trained weights? And stable diffusion I think used their clip model initially.

You should probably mention Microsoft's margin on those cloud credits. Their $10b investment didn't cost them $10b. Their return is capped at much more than 10X, seemingly using Azure cloud credits as a way of working around OpenAI's maximum profit corporate bylaws.

And if OpenAI nets enough profit for a future multi trillion dollar valuation, what is OpenAI's margin? The non margin part will partly (mostly?) be cloud bills with Azure exceeding the $10billion credits by a lot. Part of the investment was Azure exclusivity. If they become Azure's #1 customer, MS can set the pricing to whatever they want to claw back more as long as they drop their other customers (assuming this is an AGI scenario).

Basically OpenAI's profit cap is laundered and bypassed into MS cloud bills and MS is the defacto owner of OpenAI through that laundering system if OpenAI becomes huge but needs lots of compute.

> Dedicated graphics is completely idle during this.

Are you sure fixed function decoder/upscale stuff is reported in GPU utilization graphs?

Increasing linear density by 2X (not necessarily happening depending on how they are applying the marketing term to actual sizes) means quadrupling the number of transistors.

> The energy required to recognize the fruit

Have you actually worked this out? ML inference is much less energy intensive than training.

> there has not been any UBI.

The earned income tax credit (EITC) is pretty close.