This is one of several reasons the heliocentric theory took so long to catch on, despite being proposed as far back as the ancient Greeks. The existing conception of physics described above fits quite nicely with a geocentric universe, but doesn't mesh at all with a heliocentric theory. You need a whole new sort of physics (like gravity) to make sense of that.

Incidentally, this also means that Earth's position at the "center of the universe" in the geocentric theory wasn't quite as special as we sometimes think today. The earth was at the center, but the center wasn't necessarily seen as the "best" spot, it was more at the bottom of the cosmic pile, the place where all the dirt falls down to. The outer regions, aka the heavens, were often considered the "best seats" (due to their association with, well, heaven). There was often thought to be a "Fifth Element" (yes, the movie got its name from this idea) that inhabited the highest reaches away from earth and was what the stars and planets were made of.

This also means that the movement of the planets was seen as a fundamentally different sort of thing than the movement of apples. Apples were following the nature of earth, going toward the center. Planets were following the nature of their element, moving in perfect, ordered circles in the heavens.