How To: Understand Survey Methodology

It's pretty common to see polls mentioned on the cover of USAToday or on the evening news, but how reliable is that information? Today we're taking a look at survey methodology and discussing what to look for when you're trying to find credible poll data.

% error
The percent error or sampling error should be 3.5% or less. If a survey is done online, the percent error may be called a theoretical sampling error, but for the most part, percent error is about as theoretical as evolution.

Selection of participants
Whether the poll is done by phone, paper survey, or online, the participants should be randomly selected and demographically equivalent to the population of whatever group the survey is talking about. For national poll data, the U.S. Census provides a good guideline. Census data is also available for each state, which is important because age, race, and gender are not equally represented in every region.

How and when the poll is conducted
Not so long ago, credible public opinion polling was done almost exclusively by phone. Now, online polling can be as statistically significant if the participants are randomly selected. Both types of polling take about a week to reach the 800 to 1,000 respondents necessary for an acceptable percent error. When the poll is conducted can make a big difference in the results, so it is important to remember what major events were happening during the polling time. An election, a natural disaster, or a product recall can quickly change public opinion, and therefore the results of a survey.

Understanding the methodology of a poll can make a big difference in whether or not you trust the data. There are a number of other components to consider, but methodology is a quick and easy place to start. What do you look for when reviewing poll data?