When your computer first starts up, it initializes to a known good state. A lot of problems that happen in software are a result of things ending up in a bad state. By turning it off and on, you return to that original good state.

Take a game of chess for example, when you start, all the pieces are in known positions and you can start playing the game (good state). Now, say halfway through a game, someone bumps the table, and scatters all the pieces around (bad state). You don't know exactly where everything was, so the only fair thing to do is clear the board (turn off), and start a whole new game (turn on), thus returning the pieces to their starting positions (good state).