I have attended a talk yesterday where the speaker was alluding to something called "fundamental models", where the future of deep learning is centralised with large organisation owning pre-trained extremely heavy models and providing them as a service to those wanting to finetune their top layers for a specific task