Any clues what their religion was like?

The idea that the Byzantines weren’t Roman is very West European centric. The Arab Muslims in the 600s went to war against an empire which still spanned most of the Mediterranean, and was recognizably the Roman Empire. The city of Rome would remain part of that empire until the 750s, and the Empire would survive in the old Italian heartland until the 1000s.

I think there is a strong case to be made that 1204 should be seen as the end of the empire as opposed to 1453.