I'm with you right up to the turn - kind of. He flat called your preflop raise and check-called your big Cbet; you have to assume he has something, right? Over Pair, pocket pair, top pair, draw, set, something! You had...one middle pair and a draw on a very draw heavy board. What if he had an Ace? You still would have lost.
So, you check the turn, hoping to get a flush on the river. Question is, what would he have done then? Bet the river? Shove the river? Or just check, assuming you would bet in order to check raise you? You have a hand with some showdown potential but basically no more than a bluff catcher.
You want to bully shorter stacks, but this was almost 1/2 your stack lost. With nothing but a lone single pair and few outs with one card to come.
However, think about what happened...the action folded around to you and then you raised. This looks weak, right, from the HJ position? And the SB has a wayyyy wide range here. The check on the flop makes him look weak as well, but with that coordinated of a flop...see what I mean? IMO, you should have gone into pot-control mode here post turn. Checking the flop here even is not overly horrible, as it was a coordinated board. The C-bet has been grossly overused and misused frankly. And notice after he hit his set, he checked...was it because he just knew you would bet? Had you checked, he might have bet small on the river holding such a big hand as a set worried that you would not be strong enough to call too big a bet.