It is one thing to cultivate and flatter the disaffected daughter of the king, it’s quite another to stand behind the curtains when he visits his mistress.

The ancient — if sometimes less than noble — art of spying has been deployed from the time of collective settlement, to prepare for attack or pre-emptive strike.

The sophistication of available tools has expanded the reach of information gathering for all major units of power, public and private. It has become a necessary component of competitive undertakings. Diplomatic networks and embassies are an important element in the network of spying undertaken by all states, with Australia recently accused of spying on Indonesia, on behalf of the US, via its embassy. It all makes for an interesting backdrop — and some tricky questions — for Foreign Minister Julie Bishop, currently in Bali.

America will do whatever it takes to maintain its pre-eminence in the world. Information is power. Global competition is a life-and-death struggle for America — and it expects unquestioning loyalty from close allies. Australia has declared itself to be such an ally, and the price agreed upon is to do or die, most recently in Afghanistan. The diplomatic cost to Australia is increasing.

Australia now has more listening facilities across the continent — on America’s behalf — than ever before.

These are based at Pine Gap in the Northern Territory, Shoal Bay near Darwin, Geraldton in WA, a location to the east of Canberra, Cocos Island and apparently a new facility near Wagga. Pine Gap deals with ECHALON (a system for tapping into worldwide communications), drones and launching missiles. Shoal Bay listens to Indonesia and the region, and is run by the Australian Signals Directorate. Geraldton listens to and directs submarines and perhaps other naval craft; Canberra probably listens to and processes information from embassies. Cocos monitors refugee traffic and other northern ocean signals.

Former prime minister John Howard signalled to the world that Australia was the US deputy sheriff of the Asia Pacific. Recent events have confirmed his statement.

WikiLeaks demonstrated that nothing was too puerile for inclusion in American diplomatic cables; NSA whistleblower Edward Snowden has shown the extent to which the US has been prepared to go to obtain information.

Forelock-tugging has won Australia the right to spy on its neighbours; embarrassingly it has now been outed for this, in Indonesia. Did Australia’s pedestrian and mediocre spy chiefs and their equally mediocre political masters really believe they could keep secret their contracted spying? America and Americans cannot keep secrets. Time and good intent has worked to reveal some very dirty linen.

“Forelock-tugging has won Australia the right to spy on its neighbours; embarrassingly it has now been outed for this, in Indonesia.”

Indonesia would expect Australia to engage in information-gathering. If Indonesia wanted to play down reports that Australia has been spying on behalf of the US, it would have issued a statement saying it did not believe the reports; instead Indonesia has chosen to up the ante. Why? There are a number of possibilities, the first being that the matter has become public so it must be addressed. What is likely to have annoyed Indonesian officials is that Australia is apparently prepared to act on behalf of the US as a South Asian surrogate of a superpower about which they have mixed feelings. They are wary that American rivalry with China has the potential to cause regional instability, and they don’t like what they see as US hostility to Islam. Nonetheless, Indonesia recognises that the US has a legitimate role in the region. Naturally it would like that role to be open, and to be conducted by the US.

Australia looks sneaky in hiding US security interests behind the doors of its embassies. Australia has lost face, although it seems that Prime Minister Tony Abbott and Julie Bishop have not been made aware of it. Face, as they will come to appreciate, is important in Asia.

A misreading of the relationship with Indonesia also lies behind the strength of the Indonesian response. Marty Natalegawa is a good foreign minister. He understands Australian thinking and politics, which is more than can be said for his Australian counterpart. But understanding doesn’t equate with like or respect.

What we are witnessing is bubbling over of resentment that has been quietly boiling away for some time. Howard’s regional deals on boats and refugees were resented in Jakarta, but for the sake of the relationship Indonesia acquiesced.

Then-prime minister Kevin Rudd caused anger over his insistence that Indonesia accept responsibility for refugees aboard the Oceanic Viking in 2009. Abbott, Bishop and Immigration Minister Scott Morrison caused anger over statements, before the recent election, that refugee vessels would be towed back to Indonesia. These statements were conveyed in a manner that implied Indonesian compliance. It was crude, and not dispelled by the clumsy diplomacy since deployed by Abbott and Bishop.

At a deeper level the Indonesians do not like what they see as Australian exceptionalism, which translates into an arrogance they perceive as being related to racism. They believe our attitudes to refugees to be selfish. They resent that 68 years after independence, ties are not stronger, and that Indonesian culture and language are not widely appreciated in Australia. They feel they understand us much better than we understand them, and that they make more compromises in the relationship.

Indonesia is upset with us for being the stalking horse of the US in covert spying operations for information, much of which might have been obtained through goodwill if we had bothered to put some depth into the relationship.