As I say often, it's not just WHAT you say, but HOW you say it (and in this case, WHEN you say it)...
First off, I hate seeing guys walking around with their pants hanging off their azzes...but I certainly wouldn't presume to tell anyone other than my son to pull them up, since showing your underwear doesn't break any laws for indecent exposure, that I'm aware of...so really, why it would be any of your business how some other grown azz man wears his clothes is beyond me...
Secondly, even if you absolutely felt like you had to say something, WHEN did you say it? HOW did you say it? Did you wait until you got him aside and address it man-to-man, or did you just call him out in front of a group of people? I'm just wondering what your intent was in the first place...