I'd probably try digging some deep holes on the uphill side with a post hole digger, and just laying a paver or a flagstone over them. With some luck, frost heave will shove it back into place after a few freeze/thaw cycles.

No clue if it would actually work, of course.