I have hundreds of CDs and I think the format is mostly awful. They're prone to scratching, the jewel cases fucking will break, and switching discs in your player is a bit cumbersome. If you want to digitise your collection though, CDs are a good way to go. Plus you'll have a physical copy with artwork and liner notes—those can be fun to flick through.

Vinyl looks better and (for a lot of material) has a better sound, but it's very inconvenient if you want to play a song on a whim.

Bandcamp sells both formats along with digital albums and it's the best way to put money in artists' hands through digital album sales. You might find their selection too limited though, especially if you mostly listen to popular mainstream artists.

Buying digital albums or ripping CDs means that you have complete control over your digital library and can use any music playback software you like. This is really handy if you value perfectionism in your library management, but frankly if you're just starting out then you're liable to make some mistakes as you learn. There is a lot of useful free software out there for this.

iTunes has a massive catalogue and you can find nearly anything on there, but they pay artists a pittance and their proprietary library management & file format are bollocks.