Anyway, people don't usually "let" things happen to them. Sometimes it just does whether they like it or not. It's called free speech.
Someone didn't just come up and hit him in the face without any notice or warning over a shirt. There was more than likely an argument beforehand. Therefore, he let it happen to him.
Dude, you have no idea how random some fights where I live really are. This happens like once a month or so. Besides, I know it was about the shirt because the dude said something like, "That's for wearing a queer shirt!".
Besides, even if they did get into an argurment, he didn't "let" himself get punched in the face, not by your definition. An arguement would imply that he said something in his own defense.
And, even if people don't say anything back, it doesn't make them weak. Sometimes, not allowing your antagonists jibes to get to you discourages further attempts. Other times people don't say anything because of the implied threat of violence, and they know they couldn't possibly win a fight against them, or even if they could, they might get arrested/detained. Sure, that makes them physically weak, at least in comparison, but I think it can be a smart move, depending on how much they stand to gain from the pain.
Still, most people do at least attempt to verbally defend their hobby, but sometimes this only encourages the agressor. People are very illogical, and predicting their actions can be impossible. They'll tease you if you beat down their arguement, beat them down literally, or just ignore them. Some are just in it for the sport, and there's nothing, short of killing them, that one can do about it.
Most people really don't "let" this thing happen to them; they at least attempt resistance in some form. Still, sometimes it's just better to let the teasing go, as it mostly stops itself when the antagonist gets bored.
Sorry, I didn't mean to be a jerkface or anything.
