I think all of the other answers here are sort of missing the point -- you don't need bring in entropy or statistical mechanics to answer this question.

When you've been taught that objects move to lower energy states, what is meant is they move to lower potential energy states. A ball rolls from the top of a hill (high gravitational potential energy) to the bottom of a hill (low gravitational potential energy). An electron moves from far away from the positively charged nucleus, towards the nucleus, going from high electric potential energy to low electric potential energy.

This is just how forces work. A force points in the negative direction of the gradient of potential associated with it. The gravitational force points from high to low, the force of pressure in a fluid points from high pressure to low pressure -- the direction of a force is determined by the gradient of it's potential (alternatively, you can go the other way -- if you know the force vector, you can get back to the potential by integrating it).

If you're comfortable with some calculus, you can do the math for many common potentials very easily. The gravitational potential energy on Earth is U = mgy, where y is the height, so the y-component of the gravitational force is just the negative the derivative: Fg = -dU/dy = -mg. The electrostatic potential energy is U = k q1 q2 / r, if you differentiate with respect to r, you get Fe = -dU/dr = k q1 q2 / r^2. A potential energy is defined such that the corresponding force points in the direction of decreasing potential energy.

Energy is still conserved in these calculations, because the potential energy is just becoming kinetic energy -- there's no energy being lost.