You can sand over the holes, but I would use the back of the putty knife to sink in the hole first. For the size of these holes, it will be quite often be enough.

Then, simply apply mud over the holes, they are not big enough to need tape, especially if you want to go OCD-level... using tape will result in needing to mud a large area to blend in the repair. Lots of unecessary work!

You could wet sand (with a wet sponge) instead of sanding, it will give you a smooth surface akin to drywall, meaning there won't be any texture difference (or nearly any) once paint.