Labor leads in the polls — but can we trust the data?

Labor starts the 2022 election campaign in a familiar position. For months, the opposition has held a winning position in all major polls. Last night’s Newspoll marked an inevitable campaign tightening of the race. Labor’s primary dropped to 37%, but remained ahead 53-47% on a two-party-preferred basis.

But after 2019, when Labor began the campaign on the same primary vote yet managed to lose despite months in the lead, the result shocked many pundits and punters alike, leading them to lose faith in the polls.

So can we still trust the barrage of numbers we’re going to get over the next six weeks? The answer is… maybe, kind of. The key problem, according to John Utting, head of Utting Research and a veteran Labor pollster, is that people think polling is a lot more precise than it actually is.

“It’s more of an art than a science,” he said.

A key reason for that uncertainty, says psephologist William Bowe, who runs the Poll Bludger blog, is that most people reading a poll aren’t really alert to the “margin of error” associated with polling.

“Every result has a zone of uncertainty to it. You should as a rough rule of thumb treat that uncertainty around 3%, so long as the poll is from one of the reputable, well-established series,” he said.

So a Labor lead of 53-47% should translate to a two-party-preferred lead.

Still, up until 2019, polling had a pretty good track record in Australia, at least when it came to picking who would win the election. So what exactly went wrong three years ago? Utting suspects that election was an intense example of “herding” — where pollsters make statistical tweaks to avoid outlier results.

A false narrative around Labor’s dominance probably led to a 2% pro-opposition bias, with any “deviant” data buried. Malcolm Turnbull probably remained ahead when his party knifed him.

According to Bowe, another problem was a sense of hubris among pollsters. Australia hadn’t had a major polling error like the UK had at general elections in 1992 and 2015. The industry had never had to face a reckoning. That hubris led to a lack of transparency. For example, Newspoll wasn’t clear about what it used to determine preference flows and work out the two-party preferred from primary votes.

So are pollsters going to make the same mistake this time around? One improvement is pollsters are more transparent about how information is presented. Bowe also says the way polling samples are “weighted” to better reflect the demographic makeup of the electorate is now more sophisticated.

Another development is the creation of the Australian Polling Council, which creates a code of conduct for pollsters and aims to promote greater transparency. Many pollsters, including YouGov (which administers Newspoll) and Essential, are members. Resolve, which partners with the Nine papers, and Roy Morgan are not.

So if an election were held today, you could be reasonably confident of Labor winning the national two-party-preferred vote.

“Even if the polls were as flawed as last time, the Labor lead is much greater than any inherent error last time,” Utting said.

The problem for Labor is the two-party-preferred vote doesn’t decide government, and as we know, there’s no such thing as a uniform, national swing. For a majority, the opposition must pick up at least seven seats across the country.

But polling gets less reliable on a seat-by-seat basis. That’s because pollsters are working with a far smaller sample, making it harder to weight with the same degree of sophistication, increasing the likelihood of a sampling error.

After their reputation took a whack in 2019, pollsters are determined not to make the same mistake. But the biggest challenge the industry faces is a constantly evolving, existential one. Once upon a time, it was easy to reach people on a landline. Now, nearly all polling is done online.

According to Utting, that makes it much harder to track people down and get representative samples. And that means for all the work pollsters have done to get better results, this inherent challenge remains.

“The real thing about 2022 — nobody really knows if they’ve [the pollsters] fixed it. We’ll find out in six weeks,” Utting said.