This is it. You have a fan and a light wired together, on 3-way switches. Either re-wire, or leave the fan on and use controllable bulbs. Recognize that the smart bulbs might get confused if you switch off/on the fan sometimes; they may end up back in pairing mode. Not a big issue, but it can be annoying.