Yeah, you are correct to notice those contradictions. The answer is simple: the narrative that "Germany didnt mobilize until it was too late" is arguably largely false. I recommend you read Wages of Destruction by Adam Tooze. It gives a good overview of Germany's economy during the war, and it was quite a bombshell when it first came out because he debunks many repeated myths. He basically argues that Germany even at the beginning of the war was pretty close to being "maxed out" - there wasn't much more they could've done.

Its been years since I read the book, but as I recall, during the prewar years, Germany was mobilizing basically as quickly as it reasonably could. Even in these prewar years, Germany was already running into resource and infrastructure limitations - like the trains were getting bogged down and such.

For example, there is the often repeated claim that Germany refused to mobilize its women fully to get them working in factories. Problem: the workforce participation rate of German women is over 50% in 1939, which is higher than the equivalent figure for British and American women at full mobilization in 1945! (well, technically, America never actually mange to reached its full mobilization potential before war ended). I believe the figure for Britain circa 1939 was less than 33%. In terms of women, the Western allies were arguably less mobilized than Germany throughout the entirety of the war. The reason for this is, most of those German women worked in the farms (whereas the allies were able to put their women in factories - so yes, the allies technically had more women in the factories, but that isnt the full picture). Pull those women out of farms to put them into factories, and the British blockade will starve Germany like it did in WW1. Britain got a large chunk of their food overseas and America had the most efficient agriculture sector in the world, so they didnt have this limitation.

As for the oft repeated claim of Speer pulling a miracle, kinda. He did organize things more efficiently, but many of the "miracles" that are commonly credited to him were years in the making. Getting armament production setup and going is a long term process, and it just so happens that that stuff finally came online at around the same time Speer took charge, so it makes his efforts seem more impressive than it already is.

Could German industry have done things better? Yes, with hindsight, there were a ton of things that could've been improved, you could point to this thing or that thing. But you could also say the exact same things about the allies. Tooze basically argues that in the big picture, in the macroeconomic level, and considering that this is the largest industrial war in human history, Germany did their best right from the beginning, and there wasn't a whole of "slack" left that the Germans could've tapped into.