I think you might be forgetting that when the pirates are killed/flee in terror of the ghost army, their ships are filled with gondorian soldiers and men from the coast. This is the force that sails up river and joins the battle. It wasn't just 30 dudes