You say irregular, I say older regularities that have fallen out of fashion. For example, the second aorist corresponds directly to English strong verbs—they’re both Ablaut. Ablaut used to be the regular way of making past tenses; then suffixing took over.
But yes, there’s a lot of those older unfashionable regularities as cruft in the language. Greek has a lot of inflection, and a lot of sound change and paradigm restructuring; as you can see more inflections, the effects of language change are much more visible in the inflection paradigms.
It’s why it’s actually useful to know some proto-Greek and more conservative dialects (Epic, Doric), when you’re trying to learn Ancient Greek: it helps make sense of some of the oddities. Like the compensatory lengthening of -ευς -εως (which should be -ηος), or all the /s/ dropping out of verb inflections and future tenses. It does make sense, but only when you know some of the back-story.
The irregularities are rarely completely crazy (the Hittite ὕδωρ ὕδατος, σκώρ σκατός are at the upper limit); they are more often regular subsets, rather than a single pattern for the whole set. (Liquid verbs, for example.)
The cruft comes back to bite you in the ass in Byzantine Greek. Byzantine authors show off their learning of Attic, but they don’t quite know Attic, and they often regularise where they shouldn’t. The rules for reduplication in Attic make sense; the rules in Byzantine Greek are out the window, because reduplication by then was dead in the spoken language.