Friday, October 31, 2008

The Problem With Polls: Pollsters Struggle to Adapt to the New Technologies & Voting Habits

Never before have I seen polls vary so much in their results than this election. There is a reason. Pollsters are struggling to adopt to changes in technology changing technology and a changing electorate. Here are some random thoughts on polling.

First, because of caller ID and voice mail it is increasingly tough for pollsters to actually reach homeowners. As a result, the people conducting the polls have to make more calls. There could be a political difference in what people have caller ID or are more likely to pick up the phone. For example, if wealthier people are less likely to take an "unidentified" call or one that appears to be from a phone solicitor, there may be an anti-Republican bias as wealthier pollsters are voting more Republican.

Second, on a related topic, there is a big problem now that many younger voters not having land lines. Pollsters do not generally call cell phones, so they have to try to make additional calls to younger people who have land lines to make up for the difference. There well may be a difference in political philosophy between the younger person who has a land line and a younger person who has only a cell phone. One may be a college student and the other a high school graduate with a family, for example.

Third, it is clear that media outlets do not understand "margin of error." Margin of error is the way pollsters measure random sampling error. The bigger the sample, the smaller the margin of error. If a margin of error is +/- 4 points, for example, that means that each candidate's figures can be off 4 points in either direction due to random sampling error. For example, let's say that McCain leads Obama 45-40 in a poll with a 4 point margin of error. That means that accounting for random sampling error. McCain could be anywhere between 49 to 41 in the polls. Obama's number, also accounting for random sampling error, could be anywhere between 44 and 36. The media though will report that the 5 point lead is "outside the margin of error." Actually it is not.

Fourth, many polls are screening for "likely voters" when they put together their samples. That's usually the best way to poll although identifying those likely voters isn't always easy. You never know know who is going to show up at polls. Screening for likely voters though tends to favor Republicans as Republicans typically vote more consistently than do Democrats. In this election though, I'd be more trusting of a poll that doesn't screen for "likely voters" and is of "registered voters" due to the enormous turnout. That tends to show Obama with higher numbers.

Fifth, some of the polls are showing African-Americans voting 80% for Obama. Try 95%. For whatever, reason, perhaps because of difficulty identifying one's race, those numbers are not right.

Sixth, in Indiana, I'd be wary of polling that shows substantially more Republicans being polled than Democrats. The trends appear to be that the Democrats are going to match the Republicans in turnout in the state this year. The people polled should reflect that. If the polls shows 10% more Republicans polled than Democrats, for example, I'd be skeptical of the poll results and believe it has a Republican bias.

Seventh, on a related point, I'd be wary of whether the polls are taking into consideration the hundreds of thousands of new voters in the State. If it has a "likely voter" screen the pollster is looking at voting history to form the population from which the sample is taken. New voters by definition do not have voting history.

Eight, I've seen a lot of crowing about polls on some of the lower profile race. Public opinion has not yet crystallized on races like Attorney General and Superintendent of Public Instruction. They're likely to be base line partisan identification races, with candidates running a few points above or below the base line depending on how successful they are with their limited resources to get their message out. Those lower profile race polls show an enormous number of undecided voters too. In making predictions on races like that you're better off looking at the general trend of the state, and then working off of that.

Ninth, people lie to pollsters. The focus right now is on whether people are lying about supporting Obama. Some yes, my guess, and it is purely a guess, is that so-called "Bradley effect is 2 1/2 or 3%. But people lie to pollsters on a number of topics, including their race (see above where some polls show Obama with only 80% of African-American support.) As far as the Bradley effect, its influence on causing polls to be off is not as great as many of the other factors including the one below.

Finally, polls are not good at measuring enthusiasm and intensity of support. Those factors drive turnout. That's why on Election Day some close races turn into blowouts and some blowouts in the polls turn into close races. I always use the example of the 1980 presidential election. Reagan had only a slight lead in the polls, but won by a landslide. Why? Because his supporters were enthusiastically for him while President Carter's were not. Republicans went to the polls that year while Democrats stayed home. The result was not only a Reagan landslide, but the sweeping into office of scores of Republicans nationwide, Republicans who were far behind in the polls going into the election.


Anonymous said...

I was just curious given the fact that several polls give greg zoeller a four point lead what makes you so sure that he will lose?

Also I was curious about these new polls that have come out showing the guv far ahead do you know how well their sampling is? Just asking.

Paul K. Ogden said...

The AG's race will be for the most part a base-line party vote. Most people entering the polling booth won't know much if anything about the candidates and the default will be their party. I just think with the new registrations the D's picked up this year and the Obama turnout, that the base line vote will be with the Democrats. The Daniels' people haven't helped by assuring everyone they have it won rather than what is typical - overstate how close the race is in order to get your people - the Republicans - to the polls.

(Base vote is measured wrong in the media. They look at how well the Presidential candidates did in the district. What you're supposed to do is look at races in the middle or end of the ballot where people don't know the candidates. It is in those races that people tend to default to their party while in the Presidential race they tend to vote for the candidate.)

Because I start out with the assumption that the base vote will favor the Democrats, I start off with, for example 51-49 in favor of Democrat Pence. In terms of those voters who are actually voting on the basis of the candidate and the party, I'd give Pence the edge.

I don't think Zoeller's status quo message will win out over one that advocates change, especially not in this "change" climate. There are a lot of Republicans who have been very frustrated with the lack of aggressiveness from the AG's office on issues, the lastest the claim the AG didn't have the authority to investigate voter registration fraud in Lake County. That's pretty inside baseball though.

Thus, I think Pence will run a point or two above the base in a Democrat base election. The spread I think will be 4-6 points in her favor.

No, I don't know that much about the methodology on the Governor's polls. Unfortunately, polling firms don't reveal much of that information - it's a trade secret thing.