I am really enjoying the course by https://dlsyscourse.org. You build a DL library called Needle from scratch throughout the lectures. They also offer notebooks and tests to pass, so it's more than just videos.

It's also by CMU, but the videos are different than the one referenced by /u/verboseEqualsTrue, though I plan on watching that one too.

They cover transformers and other network types, but I haven't gotten that far in the videos yet. I'm still implementing an automatic differentiator.

I have found that Karpathy videos are really good too, though I'm not following them too closely yet.