Yes it can be somewhat annoying to face because it forces us to invest huge portions of our stack to play back at them - even if we are favored. At least it makes my decision fairly easy most of the time though...fold. Say we are 100 big blinds deep and someone shoves All-in right away. Everyone folds. Okay fine. They "successfully" picked up the one big blind and one small blind (antes usually aren't in play until way further into an event if at all). That is it! They risked their entire stack for this miniscule amount. The rare time someone catches a hand to play back with like AA, then the frequent All-in Shover is likely to lose their small wins and much more in that one hand.
When they keep shoving, I tend to fold a lot; we can afford to do so. In theory, we should be widening our calling range against a maniac who might hold any two cards, but I don't personally want to be flipping (or even a slight favorite) against any player this early.
Is shoving a ton early on a good strategy? Not even close! Is it the worst strategy? Surprisingly, no it isn't. At least they recognize the power of aggression (even if misused and disproportionate to their risk). This is a tougher play-style than a Tight-Passive fish. Luckily for us, their shoving is fairly easy to counter...we just have to sit back and not let the constant shoving bother us. They "win" more than they should because people get impatient at their constant All-ins and then call them down too wide.