I'm sorry that mice are in your house. As others have said, fill holes with steel wool and seal them, set traps, wear rubber gloves.

Also, keep your food in airtight containers, preferably metal. Birdseed should be kept in a metal container too. It's a PITA, but for now you probably should keep your silverware in a container too.

I love cats but sometimes they just like to play with the mice rather than kill them.