in a political season marked by nonstop polling, a lively exchange took place recently about the state of public opinion research and what to believe about all of the numbers describing the state of the race.









the context for the discussion was set by a series of national and state surveys showing donald trump gaining on or overtaking hillary clinton in the general-election campaign. it broadened into an examination how polls are produced and used in a competitive media environment.









earlier this spring, clinton enjoyed a substantial lead over trump. now, the realclearpolitics poll average in the presidential race shows clinton with a lead of just one point: 43.8 to 42.8 percent. some recent polls showed trump ahead, including a washington post-abc news poll of registered voters released a week ago.









the shift raised questions: is this merely a bounce for trump because he has wrapped up the republican nomination while clinton is still fighting a campaign against bernie sanders? in that case, will clinton reverse trump<u+2019>s gains once she has claimed the democratic nomination? do the current polls mean that the general election will be close and hard-fought? most provocatively, is there something wrong with some of these polls?









the first salvo in the exchange came from norman j. ornstein of the american enterprise institute and alan i. abramowitz, a political scientist at emory university. both are scholars to whom i<u+2019>ve gone many times as i<u+2019>ve reported campaigns and politics generally. the two co-authored an op-ed for the new york times titled <u+201c>stop the polling insanity.<u+201d>









they pointed to what they said were <u+201c>wild fluctuations and surprising results<u+201d> in recent trump-clinton polls. they also underscored how news organizations are producing polls at a rapid rate and using them to make news and generate clicks. <u+201c>too many of this year<u+2019>s polls, and their coverage, have been cringeworthy,<u+201d> they wrote.









ornstein and abramowitz took issue with a reuters-ipsos tracking poll that showed clinton with a 13-point lead on may<u+00a0>4, a tie five days later and then a six-point lead for clinton on may 15. they questioned whether opinions could have shifted that much during a time <u+201c>when there were no major events<u+201d> in the campaign.









they challenged an online nbc-surveymonkey poll that showed trump within three points of clinton and said that trump was receiving 28<u+00a0>percent of the hispanic vote when <u+201c>most other surveys have shown mr. trump eking out 10 to 12 percent among latino voters.<u+201d>









they also raised doubts about a trio of quinnipiac polls in the battleground states of florida, ohio and pennsylvania, arguing that the samples used in the surveys were <u+201c>whiter than the states had in 2012<u+201d> exit polls.









<u+201c>when polling aficionados see results that seem surprising or unusual, the first instinct is to look under the hood at things like demographic and partisan distributions,<u+201d> they wrote. <u+201c>when cable news hosts and talking heads see these kinds of results, they exult, report and analyze ad nauseam. caveats or cautions are rarely included.<u+201d>









the two scholars went on to cite well-known challenges for all types of polls. traditional polls, considered the most reliable over a long period of time, use random samples of the population, call landlines and cellphones, and use real people to conduct the interviews. but those surveys are extremely costly, and response rates for many have plummeted over the years.









online surveys use panels of potential respondents rather than randomly drawn samples. the methodology differs among the practitioners and is in a regular state of examination and refinement. they are much less expensive to produce.









ornstein and abramowitz<u+2019>s op-ed prompted a rejoinder from jon cohen, surveymonkey<u+2019>s chief research officer, and mark blumenthal, the firm<u+2019>s head of election polling. (for the record, cohen is a former polling director at the post and someone with whom i<u+2019>ve worked closely and collaboratively over many years.)









the surveymonkey duo took issue with the suggestion that polls showing trump and clinton in a close race are almost by definition to be questioned. <u+201c>it<u+2019>s not enough for trump<u+2019>s opponents to wish him away,<u+201d> they wrote. <u+201c>it<u+2019>s important for political professionals to actually explore what is buoying trump <u+2014> even if they find his rise unfathomable.<u+201d>









they argued that the polls have not been on a wild ride. in fact, they said, there was a clear trend based on the moving average of an average of all polls. once trump became the presumptive gop nominee after his victory in the may 3 indiana primary, clinton<u+2019>s lead began to shrink.









as for the nbc-surveymonkey poll showing trump winning 28<u+00a0>percent of the hispanic vote, they noted that six other national surveys taken after the reality tv star effectively secured the nomination showed his hispanic support ranging from 15<u+00a0>percent to 31<u+00a0>percent, while acknowledging they were on the high end. but they said clinton<u+2019>s margin over trump among hispanics across the six polls ranged from 23<u+00a0>points to 53<u+00a0>points. the nbc-surveymonkey poll<u+2019>s margin was in the middle of that range at 37 points.









citing the late andrew kohut, the founder of pew research, they said those in the field of survey research should be measurers not handicappers. <u+201c>now more than ever at this moment of reinvention for public opinion polling, we need many independent estimates of voter preferences, not a herd of handicappers issuing their best guesses about the eventual outcome,<u+201d> they wrote.









those exchanges prompted another voice to enter the conversation, that of mark mellman, a respected democratic pollster to whom many journalists long have gone for his insights into polls and elections. noting that ornstein, abramowitz, cohen and blumenthal were all <u+201c>very smart people,<u+201d> mellman sought to avoid taking sides and instead offered a few thoughts of his own on the issues raised.









writing in the hill, mellman began by saying that an examination of polling averages of realclearpolitics and the huffington post<u+2019>s pollster<u+2019>s model showed that there is <u+201c>little doubt that the presidential race has tightened considerably<u+201d> since march and april. but he added that the current state of the race does not necessarily mean the outcome in november will be close.









he reminded everyone that, in the spring of 2008, when john mccain had wrapped up the republican nomination while barack obama was still engaged in a hard contest against clinton, the general-election polls showed the arizona senator ahead. he lost the general election by 7<u+00a0>percentage points.









looking at the issue of trump<u+2019>s support among hispanics in the nbc-surveymonkey poll, he said that 28<u+00a0>percent <u+201c>seems high, but not bizarrely out of sync,<u+201d> given other polls and history. he also said that if trump were getting, say, 13<u+00a0>percent of the hispanic vote rather than 28<u+00a0>percent, trump<u+2019>s overall number in the horse race would be just 1.5<u+00a0>points lower.









polls have played a significant role in this campaign. they<u+2019>ve determined participation in the gop debates and how the candidates were aligned on the stage, and they<u+2019>ve driven a lot of coverage of the race. there is no question that news organizations have sometimes been indiscriminate in the way they have highlighted individual polls.









so there is food for thought in this series of exchanges. the traditional method of polling has become prohibitively expensive for most organizations at a time when the demand for public opinion surveys continues to grow, in politics and other fields. the methodology of all types of polls is under challenge. there is a serious and urgent debate underway among public opinion researchers about the way forward.









for the rest of us, the exchanges lead to common points of agreement, all of which might seem obvious but should not be forgotten. don<u+2019>t put too much emphasis on any single poll. look closely at averages of groups of polls to determine whether there are real shifts in the race. and don<u+2019>t expect polls to predict the future. leave that question to the voters in november.
a sharp exchange over election-year polling offers thoughts for the rest of us
noauthor