I believe that it all comes down to some subconscious reaction to musical intervals. Learning through modes of the major scale and you can feel this difference and get vastly different feelings.

Starting on the first note of a major scale (Ionian mode) and going through them one by one, it sounds like the Christmas song The First Noel.

Going through the exact same notes but starting the third note puts it in Phrygian mode and sounds like something from the Middle East.

The notes are the exact same, it's just anchored to a different note and starting different pattern of whole steps and half steps. Taking it further to creating full songs, songwriters are just playing off those feelings of jumps between notes (intervals) to create tension and resolution.