Sex is something you shouldn't be pushed into having. Either you want to do it in the moment, or you don't, but the second he started taking from you when you were clearly not into it, he crossed a line.

You are not having a midlife crisis. You are recognizing that you don't want what you have anymore. More, you don't have to be married to have a complete life.

Take some time and make a list of the pros and cons of staying married to him, and make your decision from there. You're not an idiot for staying in this marriage.

Say "oh, same here," then nothing else. If he freaks out at you for saying that, you know he's doing it to keep you insecure.

Then you can say "well, I don't want to keep you away from your other options," and then cut him loose.