It's funny, but there were no emperors exactly from Greece. where roman emperors were born

The ethnic groups may have been mixed. But linguistic and cultural differences persisted.

Illyria, for example, was Latin-speaking.

Even during the unified Roman Empire, the eastern half (i.e., the future "Byzantium") differed in language and culture from the western half. When Emperor Constantine moved the capital of the Empire to Byzantium, and gave a speech about it, the audience (the Greeks) did not understand his speech.

And what about the term "Byzantium", I think it is appropriate to use it to refer to the Empire between 1261-1453, because at that time very little of Greece was really Roman, because what little was left of the Roman Empire was destroyed in 1204.