I don't think you have to, for a good 90% of the books I've read, I remember the impression they left behind. I couldn't tell you the plot, or maybe I could very vaguely remember tidbits about it but I can narrate the way they books made me feel and the sentiments it covers

But anyway. I do recommend writing short essays on the books you read, I don't do it for book but I do it for films I watch. Pin down several central themes / motifs you spotted in the book and write about them. To me, the plot is secondary to the ideas a book conveys