I was the same after season 1. Dare i say it, even season 2. I really enjoyed it but i wasn't hooked until season 3, then it became one of my favourites.

It's weird though because when you later revisit those earlier seasons the quality is there, you just didn't fully appreicate what they were setting up.