It's this! For people with a lifetime listening to Western (meaning North American and European) music, your brain is just assuming "major = happy and minor = sad" because that's what you've mostly been exposed to.

But eg in Jewish music, minor keys are used all the time and don't imply anything negative. Hava Nagila is super minor and it's a wedding song that literally translates to "Let's Celebrate".

And even in Western music, that "rule" of major = happy is far from universal. Pumped Up Kicks is a dark song in a major key, SexyBack by Timberlake and Yeah by Usher are happy songs in a minor key. There's tons of cases like this, they're just not the majority. Most sad songs are minor so it's just an association we develop from past experience.