First off, “a few latin speakers” is what we have today. At that time there were so many Vlachs and Armonians that we had special laws to cover them. And they were all over the place from Croatia to the Black Sea and from Vojvodina till the south of Greece.

The “few” latin speakers boosted the populations of Slavic states and still managed to create Romania later on. That’s how few they were.

In the 12th century there are still quite a few latin speakers in Constantinople.

On top of that,

Since when are Greek and Roman culture separated to a degree that a change of language is to be considered a change of culture? As far as i know Romans and Greeks functioned in synergy for a long time.

The status of Roman heritage between Rome and Constantinople in other words Catholics and Orthodox is a political thing.

Serbs called them Romei, Bulgarians called the Romei, Bosnians called them Romei. This i know for sure.