>and I was wondering how other people deal with this

I remember things because my engagement with a show is not limited to just watching each episode. I am often on reddit discussing/theorizing after each episode. And these hours are what really embeds the details into my memory.