Going up has the right of way. https://www.rei.com/blog/hike/trail-etiquette-who-has-the-right-of-way

However, if you are descending and you are a large group, the leave no trace principles would dictate that the larger group would have the right of way because them stepping of the trail would have a bigger impact.