I’ve always heard that there were less parts in an EV and the drivetrains were less complicated. So why do they cost more than an ICE vehicle?

I think the battery is a large part of it. Since they're much less energy dense than gas they take up the entire bottom of the car.

Supply and Demand. People are willing to play an inflated price-tag for them so there is no incentive to drop prices unless an external force (competitor, depression, etc) forces the drop.

From a purely materials standpoint, batteries are the issue. AFAIK, the battery alone in a Tesla is ~$13k mostly due to how expensive the materials (cobalt, nickel, lithium and manganese) are.

Keeping EVs inflated means the majority of car buyers will still opt for ICE for cost savings.