Actually, having less layers on while doing manual labor (such as shoveling) will help regulate your body.

If for example you go outside and your body is nice and toasty in your three layers, then start shoveling and sweat jn your three layers. The moisture goes on your clothes, then gets cold with the temperature around you. Thereby making your layers a harbor for cold liquid.