Thats a size difference in my eyes, due to a complex mating behavior humans developed over time and male physical aspects is one parameter that played/plays quite a significant role, which also indicates health.. likely the major aspects in mate selection on both sides. And could you please define what you refer to when using the term size dimorphism.

Size is not a sexual dimorphism in the human species, male and female are relatively similar in size, there are actually many women taller than man. Size, which is linked to strength, makes/made a man go up in the competence hierarchy, which makes him more attractive as a mating partner, which in turn means that the genes responsible are passed on to the next generation.

Evolutionary Design: Men are built to hunt and protect, while woman play the major role in reproduction and care taking of the next generation. This diversification happens during puberty and needs to be seen in context. It does not mean women can't be stronger than man, but at the extremes its quite clear , see top level sports.