Most citizens were just rural farmers who had no reason or means to move. The only people who moved were like some priests/scribes/gentry who might have moved, but you'd have to think about the reasons why. Did they fear they would suffer at the hands of the Ostrogoths? If not, then that removes a lot of the pressure to move to the east.