True, but there's only two of you left and there's two blinds that have to be paid. The rules don't provide an exception.
Seriously, this is completely standard. There's only so many ways you can get from three players down to two:
1 - The button knocks the small blind out. Big blind gets the button and pays the small blind next hand.
2 - The button knocks the big blind out. Small blind gets the button and pays the small blind again next hand.
3 - Small blind knocks the big blind out. What happened to you happens.
4 - Small blind knocks the button out. Big blind gets the button and pays the small blind.
5 - Big blind knocks the small blind out. Big blind gets the button and pays the small blind.
6 - Big blind knocks the button out. Big blind gets the button and pays the small blind.
There's only six ways the elimination can happen so there's not really much room for "Yeah but my situation was different"
judgement calls. And in every case the player that inherits the button pays the small blind for the first hand of heads up.
FWIW the Tournament Directors' Association rule is the exact same in intent, if not the exact wording:
35. Button in Heads-up
When heads-up the small blind is on the button and acts first. When beginning heads-up play the button may need to be adjusted to ensure that, no player takes the big blind twice in a row.
These are basically the two bibles of poker rules, and neither provides an exception for a dead small blind at the start of heads up play. They just say "put the button in the right place, and the button pays the small blind". If there was an exception to that rule, they'd state it.