I haven't played 100NL, but having played 80k or so hands at 50NL I can say that you have to give this one up. Hes betting $13 (assuming the call of $4) into a pot of only $14. Your odds
here of hitting the flush don't make it worth the call. You could get fancy here (as i imagine many good 100NL players would do) and re-raise him to call his bluff, but to do that you would have to raise it up to almost half your stack. You would be betting half your stack that he is bluffing
or that you would hit your flush draw. And (as you sort of mentioned) that is assuming that your flush draw here is good and that he doesn't have a better draw.
In summation...you've only put $7 in the pot so I would feel comfortable giving this hand up.