As others have said: mixing Latin and Greek is no longer a problem; mixing English and Greek is not that much of a problem, as you can see in Category:English words suffixed with -phile
I admit: I find brandophile, a lover of brands, and foodophile, horrible (foodophile? really?). And computerphile is way too close to computer file. But I don’t have a serious objection to chocophile.
I turn my nose up at dogophile too; but is cynophile really better?
Latin–Greek hybrids though? Like paganophile or raptophile? No problem.
So, gemmophile. I disagree with Alberto Yagos: where would the –is in gemmisphile come from? And even if most of us don’t pronounce the double -mm- in gemmophile, it is there.
If you do want to go full Greek… well, make sure you do: raptophile is a pleasant word for an unpleasant condition (getting your rocks off from rape), but the fully Greek biastophile is unpleasant all round. (And wrong: it’s getting your rocks off from rapists. It should have been biasmophile.)
OK, enough preamble.
I don’t know (Ancient) Greek: I know how to look up Ancient Greek. And I assume that the Modern Greek word is wrong by default. kosmēma means jewellery now, but its original meaning is “decoration, adornment”, and it mainly referred to ornaments on dresses. Cosmematophile is also too close to cosmetics (stuff that you decorate yourself with).
kosmēma is derivted from kosmos, but the ambiguity of kosmos makes cosmophile a no-go. (Kosmos “order”, hence both “something that looks orderly, i.e. beautiful” and “order in the world = the world itself”.) And unsurprisingly, cosmophile has been used to mean cosmopolitan already.
A gemstone could be a lithos, but that is a stone in general. And lithophile is already used in chemistry: “stone-loving” metals are “elements which are commonly found as silicates and are supposed to have concentrated in the outermost zone when the earth was molten” (OED)
A gemstone could be a sphragis, though that is primarily a seal. Woodhouse’s reverse dictionary is not much better for jewellery or ornaments: Woodhouse’s English-Greek Dictionary Page Image; Woodhouse’s English-Greek Dictionary Page Image.
If you’re going the gemstone route, I’d actually go back to Latin, and do lapidophile. Lapis is ambiguous between stone and precious stone, like lithos is; but lapid– is slightly more associated with precious stones in English than litho– is: lapidary.
… or just gemmophile, which is at least not ambiguous. We already have gemmiferous and gemmosity.