Would you publish?

We know outliers are inevitable, they’re something that cannot be avoided – but from a pollster’s perspective, running a poll where your results look a little iffy becomes a bit of a two edged sword. Regardless of whether they publish or not, ultimately they lose in the short term.

If they take the “publish and be damned” approach it sets the hares, like me, running around screaming “Outlier!”. Others might start to see manipulation where none exist, questioning the polling organisation itself rather than simply the poll it produced.

Alternatively, if they withhold the poll from publication, questions are inevitably asked about “Why?”. This too sets the hares running with accusations inevitably flowing about how X media organisation is boosting Y political party by not releasing polling data that is probably bad for them. What ever decision a pollster takes, there is an automatic downside to producing an outlier that extends beyond the actual numbers themselves.

It’s a really tricky question that every pollster has to deal with at some point in time – and most come down on the side of publish and be damned.

So what should a pollster do? What would you do if you ran a polling organisation that produced a result that was almost certainly an outlier?

Is there any way you could not lose in the short term simply by producing an outlier poll that probability alone suggests is inevitable at some time?