You should keep them around. They are probably the same bricks used in the building of something in your house and are/will be useful in repairing that thing when it eventually needs it. That will give you a near perfect repair appearance, rather than trying to source new bricks that match the color and size of the originals.