This used to be more of a problem for me. I used to go back whole chapters because I was confused, but then I realized I hadn't really missed much and if I'd kept listening it would've eventually made sense. Now I'll often back up 20 or 30 seconds to the first thing I recognize when my mind wanders. Honestly I have to do the same thing when reading physical books: my mind will wander and I back up a paragraph or page or so. For me it was just a matter of adjusting and getting used to a new medium I think...