There are several factors involved including the effectiveness and numbers of the army, the health and numbers of the Roman population, the strength of the economy, the competence of the emperors, the internal struggles for power, the religious changes of the period, and the efficiency of the civil administration. All of this made the western half of the empire weak and vulnerable. They didn’t even deal with the non-Romans who crossed the Rhine, let alone those who remained on the other side.