It actually happened very often, especially if both men were similarly armed and armored. I'm gonna assume you're referring to pre-modern battles since you mention Samurais and swords.

Usually, the majority of an army was infantrymen wielding melee weapons, and in fact the word "melee" refers to the confusing, scary part of pre-modern battles where two infantry contingents finally clash and don't have much room to maneuver.

Medieval European martial arts manuals have sections on hand to hand combat, including grappling an enemy with a sword or spear with and without your own weapons.

Indian martial arts under the broad label of Kalaripayattu include armed and unarmed instruction. Same with Chinese martial arts.

I don't really know much about other parts of the world, but wrestling and grappling seems to be a fairly universal past-time. You even see Mongol and Turkic wrestling traditions, and those dudes were most famous for horseback fighting. Those skills would probably be necessary for lance-wielding cavalry if they get dismounted.