There are molecules in your eyes that undergo chemical reactions when light hits them. Those chemicals are responsible for the eye's sensitivity to light. It takes a while for them to transform back to their original state. The reason for the difference in speed is because it takes a while to go from daylight to darkness so during that time your eyes would slowly be adjusting to the change in the amount of light until it was completely dark and you went to bed. At which point you'd wake up and it would be mostly light out again and you'd need to be able to see right away.