Nobody can really answer that but you. IMO you probably over reacted, but if that event was going to put you on tilt and make you play badly then maybe it was best to leave.
Just for the sake of the exercise though...
Robert's Rules says:
Section 3 - general poker rules
15. If the dealer prematurely deals any cards before the betting is complete, those cards will not play, even if a player who has not acted decides to fold.
Section 5 - hold 'em
5. If the dealer burns and turns before a betting round is complete, the card(s) may not be used, even if all subsequent players elect to fold. Nobody has an option of accepting or rejecting the card. The betting is then completed, and the error rectified in the prescribed manner for that situation (see 8).
8. A dealing error for the fourth boardcard is rectified in a manner to least influence the identity of the boardcards that would have been used without the error. The dealer burns and deals what would have been the fifth card in the fourth card’s place. After this round of betting, the dealer reshuffles the deck, including the card that was taken out of play, but not including the burncards or discards. The dealer then cuts the deck and deals the final card without burning a card. If the fifth card is turned up prematurely, the deck is reshuffled and dealt in the same manner.
That describes the action in your hand, and it actually surprises me a little - my gut reaction would've been to leave the card on the board and proceed as though everyone had just checked the flop (and then look super-embarrassed).
The other thing to remember about this hand too is that even though there's been an error, it's an error where all the players are privy to the same information. Yes, it changes the play of the hand when they know what they turn card is going to be. But it changes the hand in the same way for all of them, which is another reason I would've been happy to let it slide if they were.
Technically though yes, you were correct to object.