Of course Frazier was no saint and Ali no devil but... we do like to cast things that way.
I can't help being reminded of Clarence Thomas, of how for a black man the ultimate insult is to be called an Uncle Tom. It's an example of how potent shaming is in the black community, and you can see how Thomas has withdrawn and simmered similarly.
From the article:
Randy Roberts says: ‘One of the many paradoxes about Ali is that he embraced an ideology that disparaged white people, yet he was never cruel to white people — only blacks.’ And Frazier, he adds, was treated most cruelly of all.
Frazier explained his feelings by saying Ali had robbed him of the gift he most prized — the American public’s respect. The cruelly inaccurate ‘Uncle Tom’ had stuck with him, tarnishing his legacy.
‘You don’t do to a man what Ali did to Joe,’ says Bob Watson. ‘People only saw one Joe; the one created by Ali. If you’re a man, that’s going to get you in a big way.’