Naw, see the work of Dr. Helen Fisher on different ways we experience love, romantic love & sexuality. You can totally love somone and have a brief sexual thought about another -- and brief sexual thoughts seem pretty common from my talks with other people. But perhaps this is not a kind thing to mention to a partner. After all, unless you have a kink around these things, most people don't really want to hear that their partner experiences sexual thoughts towards others? Having a brief sexual thought about another is not a horrible thing, it's just a human thing.