Apparently goose bumps are a relic of our ape-like ancestors. Back when our ancestors were covered in fur, the goose bump reflex would make their fur rise and make them look bigger. This was useful in scaring off predators.

It was also useful when cold, because it would create a layer of air for insulation. This reflex is common in many mammals, and is most obvious in porcupines, since they have long quills.

Now that our hair is sparse it doesn't do us much good. But we still have some vestigial body parts (like the appendix) or reflexes (like goose bumps) that used to be more useful.