If anything, the carbon footprint of training AI models have become bigger and bigger each passing year. The large LLMs are trained with clusters so big, that only the very very top tier tech companies can afford.

I think it's also about Transfer learning. Many are using fixed backbones.