I have to concentrate when listening to an audiobook so that I don't wander too far off the story.

I quite often listen while cleaning or gardening - both are things I can do without thinking. If I try to listen while doing more difficult (thinking) tasks like cooking or driving, I have a harder time keeping track of the stories.

That all said, I listen to almost exclusively fantasy books where small remarks can fill in holes or names can be confusing, so concentrating is important.