Understand Political Polling
How it works and what it means.
February 7, 2016
We’re in the midst of another presidential election campaign. Every day now, we hear that someone is X number of points ahead “in the polls”. But what does that mean? Candidates often say they “don’t believe in polls” when they’re losing. I never heard a leading candidate say that.
According to BusinessDictionary.com, a poll is a public-opinion survey in which either all members of a particular group, or randomly chosen respondents from a sector of population, are asked carefully designed questions to extract specific information.
Polling is the universal element in any political race. People often disregard it because they don’t understand it. That’s because pollsters sometimes distort their data and the media doesn’t always report it fully or accurately. But polling can be highly accurate IF it’s done properly.
The analysis of statistical trends has been around since at least the mid-17th century, but modern statistical methods emerged in the late 19th and early 20th centuries. Political polling employs a branch called inferential statistics to predict what voters will do. It’s a highly technical and exacting discipline. For a real-life example, watch the movie Moneyball with Brad Pitt and Jonah Hill. Yes, that’s a true story.
Statistics includes many technical mathematical principles such as frequency distribution, variance, standard deviation, parameter inference, and confidence intervals. We won’t go into that here. If you’re interested, there are plenty of books in your local library. These are just the basics. Let’s start with some definitions.
Statistic
A number which describes a particular characteristic, such as a baseball player’s batting average, the number of licensed drivers in a state, or the number of people who favor a particular political candidate.
Population
The entire group of people whose opinion you seek. Your population may be all voters, or all Republicans, or only Republicans who plan to vote, or only women Republicans, or all voters under age 30, or whatever.
Poll
A survey of a population seeking specific information.
Demographics
Particular personal characteristics of the population you seek, such as age, gender, race, ethnicity, education level, income level, occupation, location, etc.
Sample
Your sample is a subset of your population. If you want to know the entire population’s opinion, the best method would be to ask all of them. But that’s too expensive and time-consuming. To be accurate, your sample must be demographically identical to your population. There are right and wrong ways to select demographically accurate samples. A properly selected random sample of about 1,600 people can provide accurate results, usually with a 2.5% margin of error.
The larger your sample size, the more confident you can be in getting accurate results.
Random sample
How do you find the people who will answer your questions? A random sample is a subset of your population selected so that each member of the population has an equal chance of being chosen. You don’t choose the survey participants who are most likely to give you the answers you want.
Survey Method
How do you reach the people you’re polling? By telephone? Online? Mail? Standing on a street corner? If you only use the local phone book, you won’t reach many young people, who tend not to have landline phones. That’s been a common error in many polls. You also won’t reach many higher-income people, who often have unlisted numbers, or very low-income people, who often don’t have telephones. You must use any methods necessary to reach valid representatives of the population.
Survey Design
What do you want to know? Who the poll participants prefer as president? Or who they intend to vote for? Or who they think will win? What are their choices? All Democrats in a primary? The Democratic vs. Republican nominees? Or some other choice?
Survey Timing
Opinions and population demographics change over time.
A five-year-old survey of a college neighborhood won’t be accurate today.
Margin of Error
A mathematical formula will tell you how statistically valid your results are. That’s the small number you see at the end of every poll. If the margin of error is plus or minus two percent (+/- 2%), then a statistic of 24% will actually be between 22 and 26%.
These components combined are called your methodology. A properly designed poll will tell you exactly what you want to know. It will answer the questions that you ask. The trick is in designing the questions and asking the right people. If you want the truth, you can get it. But if you want a poll that shows your particular candidate in the lead, you can get that too. All you have to do is control the questions, sample population, and survey method accordingly.
Sampling and methodology account for variations in poll results. In political polling, you’ll always see small differences in numbers, but you’ll rarely see one poll with vastly contradictory results.
Polling Organizations
Universities, media outlets, and professional polling companies conduct most political polls. The most familiar names are Gallup, Marist, Pew, Quinnipiac, Rasmussen, CNN, Fox “News”, NBC, and Reuters. Local universities and media conduct polls on local and regional issues. They all conduct polls on other issues in addition to political campaigns. A good poll report includes a description of its methodology.
Naturally, each poll has its own website with all the data and details. But you can get a good summary of the general trend of polls through two major sites. Watch the changes over time to learn how public opinion is moving.
RealClearPolitics.comwas founded in 2000 by two self-described “political junkies” who wanted to offer a place to get the latest polling news quickly.
FiveThirtyEight.com’s founder, Nate Silver, was a statistician and writer who spent his early career analyzing baseball statistics. He created the fivethirtyeight site in 2008 to apply the same mathematical principles to analyzing the presidential election, and correctly predicted the election results in 49 of the 50 United States. Its name comes from the number of votes in the Electoral College. The site also publishes reports on sports, economics, and other categories.
I’ve heard many people complain that pollsters never seek their opinions. I’m called for polls at least a few times in every election cycle. I guess that’s because I still have a landline phone, and I answer calls even when I don’t recognize the caller ID. Life is more interesting that way.
For more information:
FiveThirtyEight.com
RealClearPolitics.com
Contact Pat to get email notice of all new Community Matters articles.
Patricia A. O'Malley
Social Policy & Programs Consulting ~ Community Matters
P.O. Box 97803 ~ Pittsburgh, PA 15227 ~ 412-310-4886 ~ info@patomalley-consulting.com
Copyright Patricia A. O'Malley ~ All rights reserved
Established 1993
Patricia A. O'Malley
Social Policy & Programs Consulting
Training and Services for agencies working toward social and economic justice