Be Wary of Polls

Polls have become very popular in recent years.  Most of them have to do with politics or current issues.  This makes sense in a democracy where politicians are always looking over their shoulders to guard against being on the “wrong side” of an issue when election time rolls around.  Polls, along with demonstrations, marches, e-mails and phone calls, have become an effective way to influence political decisions between elections.

Many polls ask people about their practices and habits, helping manufacturers, service providers, retailers, or advertisers discover our interests and how our tastes have changed.  (Is that why there are 14 types/colors of the same brand of dishwashing liquid to choose from at the grocery store?)

Since we are constantly inundated with poll results, here are a few tips.

The reason for polling is to find out what people think – all people.  It’s too expensive to ask everyone, so they ask a sample.  Professional outfits carefully plan to get a representative sample.  Others may not, and a non-representative sample, a "non-scientific poll, is pretty much worthless.  They must also get a large enough sample.  Usually around 1200 subjects will yield an accuracy of plus or minus 3%.  This is good enough, because increasing the accuracy requires many more subjects.  Return on the extra effort is marginal, and it’s still not 100% and never will be unless you ask everyone.

Wording of the questions can lead people to a particular conclusion.  Many news reports for the sake of brevity summarize responses.  Often different polls get seemingly conflicting results on the same issue.  This may be due to slightly different wording of the question.  Another wording-related issue comes with the time frame: “Has he experienced X in the last year?” or “Has he ever experienced X?”  Not knowing the precise wording can blur the picture.

I'm always wary of self-reporting.  When polls ask, do you do something or approve of something, it often leads to exaggeration, trying to please or trying to impress.  From this article about a budgeting poll we learn: “People want to be seen as good citizens, so when they’re asked by a pollster whether they do things that are generally seen as positive, good-person activities, they fudge a little. Pollsters call this ‘social desirability bias,’ and a great example of it is voting. Without fail, more people tell pollsters that they turn out to vote than actually do.”  Accordingly, they view the budgeting responses a little skeptically.  Think about how easily this bias could apply to questions on subjects like food choices, hygiene habits, drug usage or sexual activities.  (The above source also gives a good example of two similar polls with different wording.)  

Finally, do they ask facts or opinions on things that can’t be changed or don't really matter?  These are reported frequently in the news and are virtually meaningless.  An example comes from a CBS news poll: “"How likely do you think it is that a major earthquake will happen in the U.S. in the next 20 years?”  Opinions don’t cause or prevent earthquakes, but it might help some politician or marketer to play on people’s fears.  Here’s one on gun laws: "Regardless of how you feel about the issue, how likely do you think it is that the President and Congress will pass any laws that will bring about significant changes to gun policy by the end of this year?”  Do they want some psychic prediction or are they just measuring the general discouragement on progress?  Finally, “Americans are divided as to whether fathers today are more involved or less involved in raising their kids than they were 20 years ago.”  So what?

Polls can be tricky and the polltakers can use them to manipulate us, to unintentionally mislead us or just to clutter our minds with more useless information.