Greeks have a keen sense of Other, and skin colour can factor into that. As Dimitra Triantafyllidou says, we have a history of dismissing Gypsies (like much of Europe); and there’s a lot of anti-Pakistani feeling in downtown Athens. But then again, there was a lot of anti-Albanian feeling in downtown Athens twenty years ago.
But here’s the thing. Swedes are almost as alien to Greeks as Nigerians are. And a good deal more alien than the Lebanese are.
And that’s where the peculiarly American dichotomy of White/Black falls down. As Yannis Gaitanas said, “Us” is not defined in Greek by skin colour—as it was in the US, whose ruling class was a melting-pot of light-skinned cultures. “Us” is defined by culture, whether as religion or ethnicity or education. So light skin on its own doesn’t mean that much. Just because we don’t identify with Nigerians doesn’t mean we identify with Swedes more.
Oh, and the traditional Modern Greek term for blacks—which is now derogatory, but was originally just neutral: arapis. From Arab. Early Modern Greeks knew that there was this place called Araby, and that black-skinned people came from there. They didn’t put Middle Easterners in that category (such as, you know, actual Arabs). As far as Greeks were concerned, those were Turks.