On February 11, Ekos released a poll, commissioned by its media client CBC, that showed the following national federal vote intention:
- Conservative: 37.3 %
- Liberal: 24.8 %
- NDP: 14.2 %
- Green: 10.7 %
- BQ: 9.9 %
In its release, Ekos noted that: “From a Liberal (or indeed NDP) perspective, this poll can be summarised as nasty, brutish, and short. The opposition is losing touch with an increasingly distant Conservative party. The Conservatives have advanced with virtually all groups and now enjoy a 12.5-point lead (the largest since October 2009).”
Within hours of that poll being released, the Conservative “Alerte-Info-Alert” bot spat out the following:
Today, CBC released a poll with results that are inconsistent with our internal polling and other recent published surveys. In the past, pollsters have sometimes reported support for our Party that is unusually high relative to the prevailing data, only to have the anomaly corrected in a subsequent poll, giving the artificial impression of negative momentum. As always, we do not comment on polling.
That's my highlight in the Tory message: The Conservatives wanted people to know that they think Ekos got it wrong. Really?
Ekos was out their poll on Feb. 11.
- On Feb. 15, Ipsos Reid had this poll
- Conservative: 39%
- Liberal: 25%
- NDP: 18%
- Green: 10%
- BQ: 9%
- Also on Feb. 15, Harris Decima had this poll:
- Conservative: 37%
- Liberal: 27%
- NDP: 14%
- Green: 10%
- BQ 10%
- Today, we have Nanos [PDF]:
- Conservative: 39.7%
- Liberal: 26.6%
- NDP: 18.9%
- Green: 4.9%
- BQ: 9.9%
So there's four polls with broadly similar numbers and yet, the first poll, from Ekos, showed “results that are inconsistent with our internal polling,” the Conservatives said. Now to disagree with one poll and say its numbers are different than your own is one thing, but after four polls that are remarkably similar to the one Ekos had, it's pretty clear Ekos numbers did not show Conservative support that was higher “than the prevailing data.” In fact, Ekos is on the conservative side (if you'll pardon the pun). Perhaps the Conservative Party needs a new pollster. In other words, I now tend to discount the Conservative Info-Bot and now believe that, yes, the Conservative adds attacking Liberal Michael Ignatieff produced a measureable and significant bump in support for the Tories and depressed Liberal support.
But I now have some other questions worth exploring next time I run into a pollster:
Look at the numbers for the Green Party. One pollster, Nanos, has them at 4.9 per cent nationally. The others all have the Greens at 10 per cent or better. So Nanos is either missing fully half of Green party support or statistical sampling methods used by the others is counting Green support twice. And yet, while there is one major outlier when it comes to Green Support, the Conservative and Liberal numbers are all very close.
Also: Ekos and Harris Decima have the NDP in the same place at 14 per cent or , but Nanos and Ipsos have the NDP much higher than that, at 18 per cent or more. Nanos finds fewer Greens. Perhaps he found more NDP supporters? And yet, Ipsos found as many Greens as all but Nanos but has the NDP has high as Nanos.
Hi David – Nik the Pollster here,
As an FYI in the last election the Green's had a record breaking 6.78% of the vote – a high for them. We have them at 4.9 nationally.
One difference in the methodologies is that we have an open-ended ballot question which means people actually have to say Green Party of their own volition while the others prompt the Green Party in a rotation of party choices.
Another point to consider is that last week's hot air over the “problem” with polling was precipitated by the Conservative messaging about Ekos being a rogue poll – which it obviously was not. The polls overall I thought were consistent.
Cheers,
Nik Nanos
Hi David,
Have you considered the possibility that the Con-Info-Bot simply… lied? I realize its a stretch, with them being pillars of virtue and all, but still…
Also, Nanos was closest in the last election, partially on the strength of accurate Green numbers. Telling a pollster you will vote green is all well and good pre-writ, but as election day nears and voters face the reality of what kind of government will actually be formed, they will natuarally drift away from the Greens.
David – What would be considered a sufficient pool of people to poll to get an “accurate” read for a larger population? I saw the Nanos results and it would appear that only 826 people were contacted for their views. Does a count of 826 people display any accuracy for a country with a population of 34M citizens?
Just wondering.
@Nik — Thanks for jumping in an explaining the difference on the Greens.
@ToddB – I'm actually not much interested in the given accuracy of a particular poll — accuracy being defined here as: Would real election results mimic the poll results? What is more important from a methodological point of view is that the the same method is used by each pollster year in, year out. So even if you conclude that pollster A is making the same methodological error each time (i.e. sample too small or something), the fact that it's the same mistake every time means its still possible to compare trends over time from that same pollster. And that to me is far more interesting: Over the last few polls from pollster A, B, or C, have the Tories improved? Have the Liberals improved? Whether the Tories are actually at 39.7 per cent is less important to me than a pollster finding that, after applying the same methodology over several months, more people prefer the Tories and fewer prefer the Liberals.
And do I think a sample of 864 is big enough? Sure. Cuz Nik says so and he's the expert and you and I are not 🙂
Frank Graves of Ekos has some comments over at 308dotcom about the Green Party question I raised.
On the issue of the gap between GP in different polls this is a tricky problem. At the top of the list in this instance is the problem of the gap between the eligible voting population and the actual voting population . Between elections our job is to model the entire eligible populaltion . We know that only roughly 60 % will actually vote in the next election . If we knew which 60 or so would vote then we could produce results based only on the voting population (although it is still interesting and important to know how the entire eligible populaltion feesl). The problem now is that the size and compostion of the actual vs. eligble voter popultion varies from election to election .
There are always sginifcant gaps between what the population of eleigble voters say they would do and what the population of actual voters truly do .
The job of the pollster between elections is to model as accurately as possible the entire eligible population . In my view S/he should comment on possible gaps between eligible and actual voters but not try and mix up the goal of predicitng the next election with the goal of modelling the current population of voters.
So, the main challenges emanate from this porblem. If i explicity prompt for Green Party then I get a higher number than I woudl if I didn't prompt . This alone explains the main differences between what my colleague Nik Nanos finds (without prompting ) and what we find. lots of GP supporters are younger voters who dont come out as much and I also believe that eliglble GP voters are less likely to acutally vote because they know that their vote is much less likey to be rewarded with an MP than other voters (even similalry sized Bloc voters).
So we might find that 10% of eligble voters support the Green but only about 60 to 70% will show up and vote. If you dont prompt you get something lower than the actual vote , let alone the eligible voter share. This is further complcated by “others” putting themselves in the GP prompt if there is no “other” (rather than pick a main stream party ). Since 2008 we started including a prompted other because we believe it provides a more accuate model of what eligible (not the final actual) voters really want.
The single decision about how to deal with the GP explains almost all of the differemces across Nik's excellent polling and ours. Not only is GP lower in Nik's for those reasons but the “missing: GP are reallocated to the NDP and LPC (which is why they tend to be a bit higher in Nik's polling than ours
Thanks
Frank Graves
This brings up exactly what I'd mentioned here earlier, David:
http://www.thestar.com/news/canada/article/947710–conflicting-numbers-fuel-debate-over-political-polls
I'd never in my life seen a national poll for political party standings ever poll less that 2,000 people – mostly it was 2,200 to 2,400 for accuracy.
I guess if the Harper government gives you some money to poll and a list of people they'd like you to call – perhaps you'd see a 16% lead in the results?