We must face the truth: many people are too lazy to think. It is so confortable just to let the media and the mainstream culture rule you're life.
As a teen, I'm despaired by my generation. I don't know if it was the same with the former generation, but some people even manage to despise me only for being "intellectual". They see everything that doesn't fit in their alcool/MTV/handjob diet as harmful and alien. I love free jazz, philosophy and physics, so what? They see me as pendantic and self-indulgent, dangerous. What do I do? I must resign and hide my "elitist" side. Studying such matters became a kind of prohibited ritual, a kind of sabbat in Middle-Age. I'm about to build a kind of Thinking People Association, where people sharing those interest could come and talk about it with people who won't judge you.
Here is the other question: is it elitist to be intellectual? No! Being elitist means "favorising the elite", not "being the elite" as long as you don't want to be favorised. Get the picture

? When people part of a communist, anarchist or other socialist movement yell against "elitism", they yell against the people who consider plumbers to be inferiors to physicians and who agree with socially favorizing the physicians. They aren't for a intellectual witch hunt (well some are, but they just didn't get the concept). Marx, Engels...weren't they intellectuals?
It is our society which decided to favorize the "elite" as we call them and thus it is the which society built the whole "elitism" concept and its reactive counterpart, the people who are dumb and proud of being dumb. Here is how I interpret things: the elitists (not to confound with the "elite"!) took power and built a society which favorized the elite, an elitist society. The elit were very happy of how things worked, so they became elitist (exept for a few ones, like Marx and all, who were elite against elitism). Since then we confound the elite with the elitists, and the non-elit people, unable to reach the upper class, reacted by creating a new set of value for themselves. Result: some elitists are at the head of the state (happy of ruling the mass), whereas the non-elitist elite stayed with the "dumb" mass, overwhelmed by the defender of their "dumb" value (a kind of "will you ever leave us alone!" reaction).
Tadam.