The water in your water heater tank is far to hot to use, so instead it is blended with cold water at the tap. This increases the amount of hot water available to be used. So, when you turn the cold tap down, you are reducing the amount of cold water being mixed with the incoming hot water. This is why turning off your cold tap makes your water get hotter. As for why it takes so long, as another commenter pointed out, it takes time for the water that has been sitting stagnant in the pipes for a while to get used and for truly cold water to be running through your taps. The water that has been sitting in your pipes in your house is a good deal warmer than the fresh cold water coming from the underground main line.