If the markings are sitting on top of the page, you can try to remove with different kinds of professional/artist erasers (I.e. not dollar store quality).

Try kneeding eraser (roll it over, not rubbing it) a few times to see if it helps. You'll need to do it several times to lift off the dirt (if it works).

Another eraser to try is an adhesive removing eraser. You'll have to do small movements/ spot treatment to remove the first.

You can look for other polymer erasers from an artist shop.

Artist erasers tend to be better quality than regular school stationery because artists sometimes want to lift graphite off the paper as cleaninly as possible, which is what you're trying to achieve. If you choose to use an eraser, make sure to make small movements in one direction to prevent ripping or creasing the paper.

If the markings have stained the paper itself then you're SOL.