IVR Polling

In 1998 and 2000, campaigns began using a technology called IVR (Interactive Voice Response). This technology was primarily used for traditional voter contact and GOTV calls -- and also widely implemented for surreptitious "push polls." It was only a matter of time before this technology would be used for polling.

In 2002, we saw the first widespread use of this technology in polling -- mainly with publicized media polls. By 2004, the use of this technology for publicly released polling became commonplace, with several organizations and media outlets routinely releasing polling data using this technology.

This has created a lot of inquires about this type of polling.

First, it is important to understand the pros and cons of the technology.

IVR is a new and constantly improving technology in the relatively short history of telephone driven public opinion surveying. These IVR surveys typically employ random digit dial (RDD) and use a recorded voice instead of a live interviewer. Their main advantages are their lower production cost, low entry barriers, and the speed of which they can be implemented and which data becomes deliverable. Their disadvantages, however, are numerous and campaigns considering employing IVR surveys must understand their tradeoffs.

Campaigns which cannot justify or budget the cost of traditional live interview polling can opt for an IVR poll to provide basic baseline information such as name ID, trial heats and perhaps opinions of a few issues to help set the campaign’s direction or conduct simple tracking polls to make adjustments in established campaign plans. IVR polls should not, at least at this time and in our opinion, be relied upon as the exclusive rational for directing significant paid media and crafting strategic and micro-targeted field plans.

At least in high profile races, IVR surveys have a history of providing reliable data where the electorate sits on issues or in support of a candidate, but IVR Polls tend to fail as the complexity of a poll increases. Their success in lower profile and low level races is largely unknown, because there is little data publicly released for races at this level.

One example of an IVR survey breaking down is in an early vulnerability / viability survey where a lengthy informed question is asked about both or multiple candidates or to present both sides of a public policy debate. IVR polls, so far, are unable to take note of a confused participant and clarify and repeat statements. Strategic message testing is also difficult to implement in IVR polling for similar reasons. Open-ended questions are also very difficult to conduct effectively within IVR surveys, leaving out invaluable "issue concerns" that help determine the relevance of the campaign’s various messages.

IVR surveys can be conducted quickly, regardless of scale because of their ability to survey a large number of people in a very short period of time. They create a very large pool of "likely" voters and their proponents suggest that respondents are somewhat less likely to lie to please a machine (than they are to a human) on screening questions regarding civic responsibility issues like their likelihood of participating in an upcoming election[1]. Detractors of IVR surveys say that the enlarged pool of "likely" voters also relates to a response bias issue where potential respondents are more likely to refuse participating in a survey conducted by a recorded voice than one conducted by a live interviewer[2]^,[3].

We should point out that political polling is typically performed under constraints that many other surveys do not encounter. Budget, time constraints and the nature of political campaigns frequently call for adjustments to ensure reliable and credible data regardless of the survey being conducted by a live interviewer or computer.

IVR polling has some additional limitations to address these adjustments. Traditional live interview surveys are better able to speak to certain demographics on their introduction, such as asking to speak to the youngest male over 18. These factors and others lead IVR surveys to rely more heavily on weighting to bring their results into accordance with known demographics. These surveys also have a limited ability to prevent unqualified respondents from participating in the survey and there are some difficulties in validating respondents in general. These factors require IVR polls to rely on "adjustments" of the data after it is collected rather than simply getting it correct at the initial point of data collection.

Every campaign occurs under unique dynamics and every survey faces unique circumstances which create different trade-offs to consider in the decision process of whether to employ a traditional live interview or IVR survey.

There is no such thing as a perfect poll, traditional or otherwise. IVR technology is constantly improving and people are becoming more comfortable with these types of surveys as they become more prominent. Despite these advancements, it is vital to take the advantages and a few of the drawbacks of IVR polling into consideration before making a decision to utilize an IVR poll for your campaign.

The bottom line is that IVR polls should not be relied upon for the collection of comprehensive strategic data such as traditional benchmark polls, but these surveys should be considered for some basic tracking surveys and when traditional live interview polling isn't a viable option because of budget constraints.

---

[1] Kaus, Mickey. 2004. "Dem Panic Watch, Part III" Slate.com, May 2. Link

[2] Quigley, Fran. 2003. "Under-counting Julia Carson: How Effective Are Political Polls?" Nuvo.net, November 13. Link

[3] Sabin, Warwick. 2004. "Survey Says?" Arkansas Times, October 7. Link