Wav2vec2 is still sota as long as this isn’t open source it’s kinda useless lmao

It’s honestly not that difficult to keep up with the significant improvements in deep learning from a fundamental research perspective. AlphaTensor, amongst many others, is a cool application of reinforcement learning. It’s implications are huge by any standards but it’s not the discovery of a new model. Although diffusion models are also not new outside of ml, they recently started to gain attention as a new model type to train. For me, this is what’s interesting: new types of models, new ways to use SGD in a DL setup. CNNs, RNNs, Att,… are fundamental computation models. Extensions, improvements and upscaling is what I see in 99.99% of papers. GAN is a new model, VAE is a new model, actor critic is a new mode, etc. when it comes to fundamental dl research there are not many of the big ones imho.