Exploitation by more advanced civilizations here on earth comes from a need/desire for more resources. Be that gold, spices, slaves, whatever.

If a civilization is sufficiently advanced to have interstellar travel, we have nothing they would want. Human slaves are less efficient than machines. Any type of material that earth has can be gained elsewhere easier. I doubt they would care about our spices. I suppose they could just be cruel, but crossing the galaxy to kill off humans would be like a human flying to the other side planet to swat a fly.

EDIT: More like building a custom solid gold jumbo jet that can only use diamonds as fuel to fly several times around the globe exclusively to swat a fly that you read about in a cuneiform tablet written in Mesopotamia.