The private security compound is on the outskirts of Kabul, along the road to Jalalabad, a notorious strip of highway, the landscape is predominantly industrial, with shipping containers set against a string of mountains on the horizon. Several logistics companies sit behind these concrete walls — this is an industry that has enjoyed a massive growth spurt since the US-led, 2001 invasion in Afghanistan.

While Indian Gurkhas trained outside to join the company’s ranks, “Scott”, a former British soldier and now the Western head of one of the country’s leading private security firms, explains that “we don’t call ourselves mercenaries” but a reliable corporation that provided “static” security for foreign embassies, journalists, aid companies, hotels and other key assets. Launching in Afghanistan soon after the US invaded, “we survive off chaos”.

“From 2002 onwards,” says Scott, “we worked with the Afghan government because the Ministry of Interior (MOI) could not secure businesses or people and Western insurance companies insisted on using a private military company [PMC]. Internationals felt they could not trust MOI when moving province to province.”

This is the reason such an industry self-perpetuates even though President Karzai has demanded for years that these companies be replaced with the interior ministry’s Afghan Private Protection Force (APPF).

According to Scott, the implementation of Karzai’s plan this year has been “chaotic”. During our interview, he received a call from an American client who didn’t understand Karzai’s new PMC rules. “This happens all the time at the moment. For example, an Afghan is supposed to be assigned in every PMC in the country but this has never happened.”

The complicated realities of modern conflict has served as the stated rationale for this massive growth industry globally, especially in war zones since September 11. Scott offers a simpler explanation. “The Americans, British and foreign forces in Iraq and Afghanistan are not big enough to re-build nations, so PMCs are needed to fill the void. We protect contractors building prisons and schools. If the US had used more troops, we would not be necessary.”

The West has now been in Afghanistan longer than both World Wars combined. The US has spent tens of billions of aid money in the country and yet working services are minimal.

Apart from the escalating rate of civilian deaths, from Taliban and Western forces, the rise of private security armies has defined the war, resulting in numerous contractor crimes against Afghan civilians. The record of Western security firms is filled with a troubling lack of justice for victims.

Two Afghan men sit upstairs in a simple restaurant near the centre of Kabul — both have families who’ve suffered privatised violence first hand. Tariq-U-Rahman and Fahim, both from Wardak Province, explain that they’ve faced threats from three elements; the Taliban, the US army and private security companies, and were subsequently forced to move to Kabul.

Afghan firms have been hired and empowered by the US military to transport their equipment across the country. The job is to guard the convoys but they regularly establish so-called security perimeters and in the process engage in fire-fights with the Taliban, wantonly harming civilians. One of the worst offenders is Watan Risk Management, a leading company with close ties to the Karzai family that pays off the Taliban not to attack US convoys.

Fahim says his cousin, a shopkeeper, was shot dead by a Watan guard a year ago for no other reason than being in the wrong place at the wrong time. Watan admitted fault, he said, and offered $US20,000 compensation but the family is still waiting for the money. The victim’s wife and children are now struggling despite the family financially assisting them.

Fahim explains that private security companies could be necessary in other countries with more stability but in Afghanistan it had only brought “misery and violence”.

The current situation in Afghanistan confirms his scepticism. M. Ashraf Haidari, a suave, American educated senior Afghan official who is the deputy assistant national security adviser and senior policy and oversight adviser to Karzai, told me that Afghan authorities were shutting the “illegal and without licence” firms and “the new rules attempt to regulate the system”.

But several Western and local security corporations confirmed to me off the record they were still operating in the area and imagined doing so for years to come, finding ways around the new rules. Furthermore, a couple of PMCs that the Karzai government said had been shut down were still operating even if signs around their compounds were removed.

“Many embassies, for example, simply won’t trust the Afghan Private Protection Force (APPF) and will continue to rely on foreign security companies,” one said.

The supposed logic of the mass expansion of the security industry post-September 11 globally is to replace tasks the state’s military can’t or won’t do. But in a poor nation such as Afghanistan resentment built quickly, I was consistently told, when it was discovered that the Afghan army was getting paid substantially less than the private militias.

Outsourcing security isn’t the only task that has become privatised in the Western-led mission. Intelligence is increasingly collected by private companies and given to American, Australian and British forces.

Some privatised intelligence has involved the hiring of corporations to gather information about Afghans that is then used by the military for so-called counter-insurgency. Jeremy Kelly in the London Times first published extracts in late March of extensive documents by US-based “consultancy company” AECOM — the company had been hired by NATO to spy on mosques, universities and the general community throughout the country. The work started just over a year ago.

I viewed dozens of pages of this intelligence (and extract below different sections to the Times). The files detail conversations from March 2012: people complain about the Karzai government’s corruption and inefficiency; clerics in mosques demand Western forces leave immediately; family members complain about proposed marriages between the Taliban and local girls; others express support for the insurgency and complain of troubles when working in Iran.

The research comes from a range of districts and is separated between “supportive” and “non-supportive” individuals of the NATO mission.

One entry, from March 14 in the Sheberghan District, details an “overheard conversation between two Uzbek males between the ages of 40-45 at market.”

“One man said, ‘The other day I was riding on a bus when it became very windy. It seemed as if it was raining dust. People were saying that this could be a sign God’s wrath. This is happening to us because the Americans have burned the Quran, but we are calmly sitting idle. We should be rising up against the Americans for what they have done. We are being punished for doing nothing.’

“The other resident stated, ‘I do not know, but it might be possible’.”

In another extract, from March 15 in Shahr-e-Safa in a public car, an Afghan spy overheard “two concerned men ages 50 to 60, discussing private escort companies threat to the safety of civilians.”

“The first man said, ‘People distrust the private escort companies because when a Talib fires at them, they return fire at houses, people, even the trees are cut if a Talib is shooting from behind them!’

“The second man replied, ‘Most of the time, innocent people are killed or injured in the crossfire.  People want the government to either make sure escorts do not harm civilians or disarm them’!”

Such details appear as mundane, normal and daily conversations by local villagers across the state, but they can form the knowledge for US-led night-raids that cause deaths and deep Afghan anger. Mistakes are routinely made. Innocent men are kidnapped. Many are killed.

The recent announcement that Afghan forces would now take the lead on night-raids was dismissed as propaganda by sources in Afghanistan, a face-saving exercise by the Karzai government to show it has sovereignty in its own country.

Meanwhile, the US military and its allies have little idea of the agendas of the Afghans giving them intelligence. It’s why respected organisations such as The Afghanistan Analysts Network refuse to undertake commissioned work for clients, concerned that its research may be co-opted for military means.

As soon as the Taliban was toppled in 2001, Northern Alliance forces and its friends routinely issued payback against enemies, real and imagined. Even today, a local warlord and police chief in Uruzgan Province, Matiullah Khan, is using Australian forces to take out his rivals and fuel conflict.

A reporter from the Chicago Tribune witnessed this trend as far back as November 2001.

Western forces enabled this behaviour by using provided intelligence and arresting, bombing and interrogating people they were told were Taliban. In reality, the information was often wrong. Crucially, it reinforced the Western belief that any breathing Taliban should be a dead Taliban.

That was then. Today, the US government realises it will have to negotiate with the Taliban but is hiring private firms to better understand who should be targeted first.

Privatised security and intelligence is now a natural part of Western war making. America simply cannot and will not launch missions without the backing of often unaccountable companies that complement its defence industry. Since the departure of US troops from Iraq, thousands of foreign contractors still populate the country, that doesn’t look set to change any time soon.

*Antony Loewenstein is an independent journalist and author who is currently working on a book and documentary about disaster capitalism.