>they didn't start a war, invade anyone or killed masses of people

They would if they could. It's not wrong to call fascists fascists just because they're not in power.