With the ABS releasing the May employment data tomorrow via its monthly Labour Force Survey, it might be worth pulling the rug out from under the inevitable stupidity that has accompanied the media commentary on the figures lately.

Remembering back to the last data released for April, the ABS had unemployment reducing from 5.7% down to 5.4% – leading to the ruination of a planned narrative so complete that it has only ever been surpassed in magnitude by the collapse of Turnbull’s “Rudd Recession” line.

Rather than deal accurately and professionally with the volatile nature of monthly employment figures – especially in a time of economic uncertainty when such figures generally produce even more volatility than usual – the pundits started grasping at straws for any excuse as to why their silly point forecasts and predictions of gloom didn’t amount to a hill of beans.

So we got the tawdry spectacle of people that should know better blaming the reduced sample size of the ABS Labour Force Survey as the culprit. Not because it was the culprit mind you – far from it, but it got blamed because the alternative for these folks was to admit they were, basically, clueless and had been caught out.

Uncertainty wasn’t a word in their lexicon – their prognostications had until this point been delivered with a certainty approaching biblical edict. The moral of the story here is that ones prognostications shouldn’t be any more certain than the data and the prevailing environment.

Rather than repeat what we said about the April figures, let’s go instead to a transcript from the Senate Economics Legislation Committee’s Budget Estimates hearings from June 2nd – last Tuesday night. Senator Hurley has the Chair (in blue) and Peter Harper, the  Acting Australian Statistician, is in red.

CHAIR—I might start off the questioning tonight. There is a blog that I have some interest in, the Pollytics blog, who are talking about some results, and I thought I would get your comment on this. It says:

Remembering back to last month when the unemployment figures were released by the ABS showing a fall from 5.7 to 5.4%, the howls of incredulity from economic firms that guessed wrong on this was deafening. The ABS unemployment figures are actually derived from a poll, albeit an enormous survey of around 41100 people, and it was pretty much lambasted across the economic sector as a “rogue” result and a consequence of the ABS reducing their sample size. The reporting of the issue was just mind numbingly dodgy.

So the poor old ABS got slapped around for weeks on how their small sample size of 41100 was creating all this enormous volatility and uncertainty in their figures, and as a result of this hysterical (and let it be said, mostly ridiculous) gnashing of teeth and stomping of tootsies by private sector economic firms – the ABS has now restored their Labor Force Survey to run on a sample of around 54400.

The blog continues—and I will paraphrase this a little:

Let’s get something straight here – this new sample size will make— no difference whatsoever.

Could you comment on the accuracy of that report?

Mr Harper—Okay. There are a lot of things in that statement. In terms of the factual side of things, yes, the ABS Labour Force Survey in recent times has had a sample of about 41,000 households. Because of budgetary pressures, the ABS reduced the size of its labour force sample in June last year from 54,000 to 41,000.

Because of additional funding that we received in the budget in May, we will restore the sample size back to the 51,000. We will do that progressively with the aim of having it fully restored by December 2009. The reduction in sample size has created an additional margin of error around the survey results. Every sample labour force survey and every other sample survey that the ABS conducts has a statistical error associated with it. That statistical error can be measured, and the ABS publishes information about that error. The reduction in the sample size for the labour force survey increased the statistical error by approximately 15 per cent. In terms of the estimates for Australia, particularly employment and unemployment, the standard error was still relatively low with the reduction in the survey size. Nonetheless they are there, and they need to be taken into
account in interpreting the results.

It is important to note that the reduction in the sample survey did not introduce any bias into the results, so the result for the month of April of 5.4 per cent was an unbiased result, just as the result for the previous month of 5.7 per cent was also an unbiased result. We checked the April estimates extensively to make sure that they were as accurate as we could make them, and we could find no reason to be concerned with them. We do know that when economic circumstances are changing, there is quite often volatility in the labour force survey results. We have gone back and looked at history and we have seen similar sorts of ups and downs in previous times of economic challenge. The other observation is that the ABS would always encourage users to look at the trend which smooths out the month-to-month volatility in the estimates and provides a longer term perspective as to what is happening with those statistics.

CHAIR—Thank you. Just to confirm, the blog says:

If we plug in 41100 as the sample size and 5.7 as the result for Poll 1, the margin of error on this survey comes out as 0.22%.

If we plug in the larger sample size that the ABS used to run and will now do so again, a sample size of 54400 with the same result of 5.7 as Poll 2 – the margin of error is 0.19%.

All this outrage has been over a structural improvement of 0.03% in the accuracy of the raw results!

Mr Harper—That is the 15 per cent that I talked about, the difference between the 0.19 and 0.22 per cent.

CHAIR—Therefore, to a layman, there is very little difference between the actual results?

Mr Harper—The reduction in standard error does lead to a more accurate result and therefore users can take somewhat more confidence in the statistics that are produced. But, as is pointed out, even with the return to the previous sample size, there still will be a margin of error around that survey. You need to make a judgment as to the fitness of purpose for the statistic, taking into account the standard error and the particular application that you have for it. It is always a trade-off that the ABS has to make in terms of the sample sizes that you have for collections which, in an ideal world, you would want to be as large as possible to minimise that error, but obviously you need to take into account the costs of conducting surveys, both in terms of the cost to the taxpayer and the cost to the providers of having to provide that information to the organisation.

CHAIR—Thank you. In other words, it is much ado about nothing, and I am pleased to have one of my favourite blogs indicate it.

A big hello goes out to the good Senator, and a wise paraphrasing it was 😀

The smaller sample size the ABS used in April produced an increase in the margin of error for the unemployment result (we’ll deal with it in terms of MoE’s rather than Standard Errors since we’re all across the nature of an MoE – this being a polling blog) that was so small that it didn’t make a material rats arse of difference – something that the ABS acknowledges.

So when the unemployment estimates come out tomorrow – regardless of their value – any pundit that carps on about reduced sample sizes needs to be treated like the amateur hour analyst they would have to be to say such things.

The data is volatile because employment is volatile – not because of sample size issues. The tight labour market conditions only recently experienced by employers are still seared into their memory and is influencing their behaviour. Some industries are as tight as they’ve ever been while other industries far less so. Some companies like retail are experiencing bursts of strong activity followed by lulls; while others still have faced nothing but poor economic conditions for over 12 months.

As a result of the wide variety of economic circumstances experienced by firms across the country, the employment data is volatile – simply because conditions are volatile.

Unemployment is a lagging indicator, so it takes a while for deteriorating economic conditions to flow through to deteriorating employment conditions. There is little doubt that unemployment will increase over time – at least until growth can pick back up above the 2.x % mark that it takes to start absorbing new entrants to the labour market. But we shouldn’t expect the unemployment level to dramatically increase in a linear fashion. The change to employment statistics will continue to be lumpy over any arbitrary period of time – we’ll have large and small monthly movements, probably in both directions over the next 12 to 18 months, simply because that is what usually happens.

So fear not the figures tomorrow. A 6.2% unemployment rate doesn’t mean the sky is falling anymore than a 5.2% rate will mean the economic downturn has disappeared. If it is a high result, it will be interesting to see if the pundits that got it wrong last time start to claim vindication and continue to erroneously blame the ABS for their own shortcomings– since they’ll be making exactly the same mistake as they made last time, albeit this time in the opposite direction.

As always, the ABS trend estimate is the one to pay attention to – particularly the adjusted trend estimate we’ll see for April (rather than the latest trend estimate for May). That adjusted April trend estimate will give us the best idea of where we are at, albeit a month behind reality. This comes about because the trend estimate for May is itself a projection containing large amounts of uncertainty, while the April trend estimate will have more certainty by being fitted to both past seasonally adjusted results (the months leading up to April) and a future seasonally adjusted result (the May result).

Tomorrow when the data is released, we’ll have a laugh at some of the commentary and run the numbers through The Poll Cruncher to take a squiz at some probabilities regarding the real movements of the unemployment rate.