Andrew Forrest is currently talking up the achievements of the Australian Employment Covenant (AEC), the scheme originally billed as seeking to create 50,000 jobs for indigenous Australians within two years.

In October 2010 — two years after the AEC was officially launched — myself and Dante Mavec examined the outcomes the scheme had achieved in that period. We were concerned about the lack of data available to assess the AEC’s merits, as well as the lack of transparency about the amount of public funds committed to the scheme.

More information has since come to light. According to the AEC’s website, employers have so far pledged about 45,000 jobs under the scheme. Forrest has said he is “quite confident” his target of 50,000 job promises will be met by the time the funding agreement with the Commonwealth government expires at the end of June.

But getting 50,000 job promises from employers is very different from getting 50,000 indigenous people in jobs. The AEC says the goalposts have not been shifted. It says the scheme was always premised on a three-way agreement, with the AEC aiming to build employer demand, employers committing the jobs, and the Commonwealth government training the indigenous people to fill them.

Exactly how many jobs have been filled is difficult to know. The AEC suggests that it’s about 4300, but because of complex difficulties with reporting, only a smaller number of these can be individually verified. Questioning in Senate estimates revealed that in February 2011 this number was just under 1800.

Understanding how many of these jobs have been filled as a direct result of the AEC is even more difficult. Some employers who are participating in the scheme had already developed Reconciliation Action Plans or been involved in Corporate Leaders for Indigenous Employment Projects. These schemes — that predate the AEC — encouraged employers to develop and implement strategies to increase indigenous participation in their workforce. So it is quite possible that they would have employed more indigenous people irrespective of their engagement with the AEC.

If the needs of indigenous job seekers are now being better met, knowing which scheme contributed to that outcome may not be important. But isolating the effects of the AEC is useful if we want to know whether it is an approach that should be extended. It is also important if we want to assess the value for money of public investment.

Senate estimates has helped us there too. The funding provided to the AEC by the Commonwealth government has included a grant of just over $4 million to help establish the organisation. But the agreement also allows for outcome payments to the AEC on the basis of three things: job promises, job placements and job retention to 26 weeks.

Because not all of the 4300 job placements identified by the AEC are individually verifiable, the organisation has only been able to claim outcome payments on a subset of these. A rough calculation on the basis of the figures revealed in Senate estimates suggests the AEC would have been able to claim about $290,000 for the 1761 job placements and $31,000 for the 286 job seekers who had remained in employment to 26 weeks. Additional outcome payments for job promises and placements secured before July 2011 — and for retention of those jobs to 26 weeks — can be claimed until March 2012.

Interestingly, though, the AEC has been able to claim much more on job promises. Again, making a rough calculation on the basis of job promises listed on the AEC website, the organisation would so far have been able to claim about $3 million in outcome payments for these pledges. Andrew Forrest’s concerted last-minute efforts to boost the number of job promises before the funding agreement’s expiration has not only given him a positive media story, it has also substantially increased the outcome payments the AEC is able to claim.

This is not to suggest any impropriety. Indeed, there is no indication that such outcome payments are used for anything other than reinvesting into AEC operations.

Nonetheless, the ability to claim outcome payments on job promises would appear unusual, particularly because the AEC acknowledges that they may never know how many of these promises turn into actual job offers, let alone jobs for indigenous people.

The number of jobs currently listed as available on the AEC’s online Jobs Board is about 900, with 5000 more forecast to become available in the next 12 months. That leaves perhaps 35,000 jobs that employers presumably plan to offer in subsequent years. But, employers even with the best of intentions may have pledged to take on a certain number of indigenous job seekers only to find that a change in their circumstances means the expected vacancies do not arise.

It is not clear that outcome payments for job promises are a wise investment in this context.

Forrest and the AEC are right that we should be looking at every available option to increase indigenous employment opportunity. But we should maintain some analytical scrutiny along the way.