The advantages of classical liberal market cosmopolitanism–the idea that it's best to set aside peaceful differences of opinion and creed and worries about different races, nationalities, and genders when deciding how we interact with the world–has a great track record of making us all richer and happier.
The idea that that people should be punished with boycott or losing their jobs over having wrong beliefs hobbles the flowering of tolerant classical liberal market cosmopolitanism.
There may have been a good reason why classical tolerance of expression was summed up in the epigram: "I disagree with what you say, but will defend to the death your right to say it!"
That has a different feel than: "I disagree with what you say, I think you are evil for having said it, I think no one should associate with you and you ought to lose your livelihood, and anyone who doesn't agree with me about all that is skating on pretty thin ice as well, but hey, I don't think you should be arrested for it."
A stern insistence on boycotting or refusing any truck or barter with those who hold different beliefs or practice different ways of life (peacefully) does not directly implicate specifically libertarian questions about rights or freedom. No one's freedom in the true libertarian sense is harmed by people trying to drive them from society or the market because of their beliefs or creed as long as it is done through mere refusal to associate, or advocacy of refusal to associate. We have no right for others to do business with us or to tolerate our beliefs or practices as long as said intolerance does not turn to violence.
But regularly acting on the idea that those with wrong ideas deserve to be driven from society in any conceivable non-violent way might, I suggest, make for a less lovable, rich, and peaceful world. When we start regularly restricting people's opportunities in commerce or association over differing beliefs, what could be peaceful ideological differences start to tip over into people fighting for what they can understandably see as their metaphorical life–their social or economic life. It's a dangerous game and if pursued vigorously and across the board by everyone who disagrees with everyone else on issues or practices they consider vital, will make everyone worse off.
Centuries after the Enlightenment, most people's notions of "free thought and expression" still amount to: it's OK to think and express OK things. It's a limited view that can lead to a less varied, vital, and livable culture.
Jonathan Rauch wrote on these issues of true liberal tolerance of differing opinion in the December issue of Reason