The actual formula for calculating switching energy is approximately E=C * Vdd^2 * fs, where C is the load capacitance (larger the transistor higher the C), Vdd is the operating voltage and fs is the switching frequency.

This means that increasing the frequency affects the power consumption at the same rate as decreasing the physical size of the transistor lowers it.

Ofc making transistors smalles is extremely difficult at the moment, which means that the increase in switching frequency pretty much negates it. This combined with the increased transistor count pretty much guarantees that higher performance -> higher power consumption.