As Names of the Greeks – Wikipedia details, the name that the Byzantines gave themselves, and the name that Modern Greeks traditionally gave themselves as a result, was Romans: Romioi, with Hellene reserved for the Ancient Greeks (or for pagans in general).
It follows that the name Greeks traditionally gave their vernacular was Roman, Romeika, with Hellenic reserved for Ancient Greek. And this term entered 18th and 19th century English as Romaic.
In fact this book written in 1855 explicitly contrasts Romaic and Modern Greek: Romaic and Modern Greek. Romaic is what Greeks called Demotic, the spoken language; “Modern Greek” is what Greeks called Katharevousa, the diglossic written language.
Romaic passed out of usage by the end of the 19th century in English, as Demoticism gained ground and as Greeks grew uncomfortable with anything Greek being called anything but Greek. Romeika is obsolete now in Greek too, though many will still remember when people used to say “am I speaking Romaic to you?!” (same meaning as “English, m*thaf*cka, do you speak it?”). That’s “am I speaking Hellenic [Greek] to you?!” now, of course.