Also they were used.

https://m.youtube.com/watch?v=CxlRJsQ7p2k&t=105s

(Yes the battle from this movie is 17th century, but they were used in similar ways earlier too.)

The big problem is black powder. That shit lights up like no one's business, and so in a musket line... They would be dangerous.

So they were really more of a sometimes weapon, especially when cannons breached a wall grenades would be used to help gain a foot hold in the breach. But the point remains people didn't walk around with them. They were provided when needed for a particular task (until they became reliable, and fit with the infantry doctrines being used ... ... Which took a while