The government’s draconian metadata retention laws come into effect today. We republish Bernard Keane’s guide to everything you need to know about these laws (and how to get around them).

Crikey has been covering data retention for several years, and we’ve written tens of thousands of words in that time explaining what it is, why it’s important and the threat it poses to Australians. As Australia’s biggest ever mass surveillance scheme gets under way, this Q&A provides a one-stop document for what data retention is, what it means for you and your country — and how to evade it.

What is data retention?

The compulsory retention of information about a citizen’s telecommunications and online usage, either by telcos and internet service providers themselves or by a government agency, so that law enforcement and intelligence agencies can use it to investigate crime and national security threats. The Australian version will force telcos and ISPs to retain your data. The Act establishing the scheme is here.

What sort of data is being retained?

Remarkably, we still don’t exactly know — despite the department that is in charge of data retention, the Attorney-General’s Department, having worked on the issue for at least eight years. Amendments to the legislation (you can see them here; the relevant section is 187AA) provided a little more clarity than we had when the bill was first introduced (which was none): the data is to consist of subscriber or account holder details, the source and destination of a communication, date, time and duration of communication, location and what services was used e.g. voice, SMS, social media, Skype, and the type of delivery services (ADSL, Wi-Fi, VoIP, cable, etc). It will not include browsing history. However, even as the legislation goes into effect, many companies caught by the scheme still have not been able to find out the specifics of the dataset they are required to retain beyond those broad categories.

What will it cost?

Again, no one is sure. The government allocated $131 million to partly fund the scheme ($3 million of which AGD took for itself), which then-Prime Minister Abbott suggested might cost $400 million. In evidence to the Joint Committee on Intelligence and Security that considered the issue in 2012, iiNet said it might cost $5 a month for every customer to store data; that would be a $60 a year surveillance tax on every household. In 2014 iiNet significantly increased its estimate of the likely cost to $130 a year. Remember, both companies and government agencies will not merely need to store this data, but ensure it is stored safely — the vast trove of personal data that data retention will produce will be immensely attractive to criminals (in 2012, Anonymous hackers released customer data obtained from AAPT to protest against the then-government’s data retention proposal). The alternative to expensive, highly secure storage is storage with a cheap offshore provider where your data can be easily hacked.

What happens currently?

Traditionally, telcos have retained phone records because that was how they billed you. The government claims companies have less and less need for metadata beyond the billing cycle, and given there’s a cost to storing such data, they are keeping less of it for the sort of time periods agencies prefer — usually two years. Law enforcement and intelligence agencies call this “going dark” — losing access to phone information of the kind they’ve had for decades.

However, major telcos like Telstra had rejected this argument and said they have no plans to abandon current retention arrangements. Moreover, both Telstra and major ISP iiNet say they will have to create new classes of data in order to comply with the law. This is not, strictly speaking, a “data retention” law but a “data creation” law.

Isn’t this just maintaining the status quo, if phone records have always been kept?

No. Your phone data now includes your location as your mobile phone interacts with nearby phone towers, so it can be used as a tracking device. But more importantly, forget that “it’s just metadata” or “billing data”. A single phone call time and duration won’t tell anyone much about you. But in aggregate, communications data will reveal far more about you than content data. With data retention, agencies can accumulate a record of everyone you have called, everyone they have called, how long you spoke for, the order of the calls, and where you were when you made the call, to build a profile that says far more about you than any solitary overheard phone call or email. It can reveal not just straightforward details such as your friends and acquaintances, but also if you have medical issues, your financial interests, what you’re buying, if you’re having an affair or ended a relationship. Combined with other publicly available information, having a full set of phone records on an individual will tell you far more than much of their content data ever will.

And if you don’t believe us, ask the people who know: the General Counsel for the United States National Security Agency has publicly stated, “metadata absolutely tells you everything about somebody’s life. If you have enough metadata, you don’t really need content.” According to the former head of the NSA, Michael Hayden, the US government kills people based on metadata it has accumulated on them. As Edward Snowden says: “You can’t trust what you’re hearing, but you can trust the metadata.”

But as things stand, it’s easier for the government to get access to communications data about you than for it to get content data about you, which requires a warrant.

So why not make government agencies get a warrant?

A number of countries actually require police forces and security agencies to get a warrant before they access communications data: Denmark, Greece, Luxembourg and Spain, just to name a few. The only reason advanced by Australian police forces and security agencies as to why they should not be required to get a warrant to access retained data is that their operations would “grind to a halt”. However, law and order seems to function perfectly well in those countries with a warrant requirement. One AFP officer said they had estimated the cost of needing to get a warrant to access data at $25 million a year. Sounds expensive – until you realise the AFP annual budget is over a billion dollars.

OK, but we’ve already given away our privacy to Facebook etc, haven’t we? Why shouldn’t agencies that want to protect us get the same data?

This is an argument routinely used by data retention advocates. But going on Facebook isn’t compulsory. Citizens choose to use social media or other online platforms and voluntarily engage in the swap of privacy for services that so many applications are built on. Maybe they don’t understand the full nature of what they’re losing in that transaction, but it’s still voluntary. There is nothing voluntary about data retention — not unless you want to withdraw from the 21st century and not use telecommunications and online services. Moreover, you might not like Facebook, but it can’t arrest or jail you using the information you put on it.

But agencies say they need it to help prevent and solve crimes.

In February 2015, a Dutch government agency found that data retention in the Netherlands had been unnecessary despite over four years of use. A German parliament study concluded data retention in Germany had led to an increase in the crime clearance rate of 0.006%.  Danish police, who have a much wider metadata and content data retention scheme, said the sheer amount of information they had was too unwieldy to use. Barack Obama’s handpicked NSA review panel found that mass surveillance by the NSA had not been necessary to stop any terrorist attacks in the United States. Australian police forces and the Attorney-General’s Department officials themselves admitted they could produce no evidence beyond anecdotes about what benefit data retention would bring, and said metrics such as crime clearance rates wouldn’t ever show any benefit.

But such-and-such a high-profile crime was solved with metadata.

Maybe – or maybe not. One case, the 2005 Operation Pendennis terrorism case in Victoria, is frequently cited by data retention advocates as an example of how communications data is crucial to fighting terrorism. But in fact communications data was unnecessary to identify either the ringleader of that group or those who were associated with him, as they conducted many of their activities in public. On other occasions, metadata has been available to assist police without a data retention regime. Since 2012, there has been a data preservation scheme available to law enforcement and intelligence agencies that requires communications companies to preserve the data of an individual — but ASIO barely bothers to use it.

But let’s assume for argument’s sake, despite the evidence, that data retention could help improve crime detection rates. In Western societies, we have long accepted that there is a trade-off between the rights of the individual, including a right to privacy, and the state’s power to protect its citizens. We understand that all our civil liberties make it harder for the state to prevent, detect and punish crime. Nonetheless, we value them enough to keep them anyway. Data retention alters this balance in favour of the state — in effect it says that we should throw away the traditional balance between civil liberties and the desire of the state to detect crime.

So why do they want it if it doesn’t work?

The international experience of the “Five Eyes” intelligence networks of the US, the UK, Canada, Australia and New Zealand is that they want “all of it” — every single piece of information they can vacuum up from any source, whether it is useful or not. And while the intelligence agencies that compose the Five Eyes network insist they are about national security and preventing crime and terrorism, in fact much of their activity is aimed not at protecting citizens, but at commercial espionage in which foreign governments and corporations are subjected to surveillance and the information passed on to Five Eyes governments and in turn to corporations in order to exploit. From this point of view, there is no reason for any limitation on data collection.

The Five Eyes mindset is one rooted in the analog era, when telecommunications were controlled by governments and security and law enforcement agencies had unfettered access to and complete control of communications infrastructure. It was also much easier to control the media, which consisted of a few radio, TV and newspaper outlets controlled by a handful of large companies. Data retention is part of an attempt to import this model to the internet, so that they have the same access and control of the online space that they had of the analog telephone network.

But we can trust our agencies to do the right thing!

Australia’s agencies generally have a better record on not abusing their powers than foreign agencies. However, ASIO, the Australian Federal Police and the Australian Secret Intelligence Service are by no means perfect: ASIS, which has bribed Indonesian people smugglers, broke the law by bugging the East Timorese cabinet in 2004; ASIO tried to intimidate and gag the former ASIS officer who, on the recommendation of the Inspector-General of Intelligence and Security, revealed the bugging in 2013. We also know from Edward Snowden that Australians intelligence agencies use electronic surveillance not for protecting us from terrorists, but for economic espionage.

The problem is that, unlike normal government bureaucracies, intelligence agencies have minimal public oversight or accountability (no investigation of ASIS’ law-breaking has ever been undertaken, for example), and can use national security as a justification to resist media scrutiny. The lack of oversight means incompetence, corruption, mission creep and criminal activity are far less likely to come to light than in normal government agencies. Public transparency is one of the key motivations for public servants to behave appropriately, and it doesn’t exist for agencies engaged in surveillance. And the more personal data they have access to, the greater the temptation.

But if you’re not doing anything wrong, you have nothing to hide.

Wear clothes in warm weather and have blinds in your windows? What are you hiding?

Are you happy for everyone to know where you are all the time, who your friends are, with whom you’re having a relationship, everyone you call, whether you have a medical or financial problem? It is not up to privacy advocates to “prove” the right to or importance of privacy. All governments acknowledge it is a fundamental right. If you support breaching that right, it is up to you to make the case, not demand privacy advocates defend it.

And law enforcement and intelligence agencies don’t merely target people “with something to hide”. People as diverse as whistleblowers, journalists, politicians, non-government groups and activists are subject to surveillance by such agencies, despite not having “done anything” other than reveal wrongdoing by governments and companies and protest against it. Data retention thus indirectly threatens core processes of democracy like whistleblowing, political organisation and scrutiny of governments. And you may be comfortable with the current government having access to your data — but what about all future governments?

Why are journalists so upset?

Data retention significantly increases the chances that governments will be able to track down whistleblowers or sources for news stories. We know that the Australian Federal Police routinely investigate journalists who have revealed public interest stories about, for example, the government’s treatment of asylum seekers, in order to track down their sources. The AFP has admitted that it obtains journalists’ metadata to do this. More metadata, over a longer period, means more risk for sources. Sources don’t even need to call a journalist — merely failing to turn your phone off while travelling to a meet a journalist could be all the evidence the police need to track someone down.

What’s been done to protect them?

In a belated amendment to the legislation, under a new “journalist information warrant”, agencies other than ASIO would not be able to obtain retained data for the purpose of identifying a journalist’s source without a journalist information warrant from an independent issuing body such as a judge, which could only be issued if:

“… the public interest in issuing the warrant outweighs the public interest in protecting the confidentiality of the identity of the source in connection with whom authorisations would be made under the authority of the warrant”.

As part of that process, the submissions of a public interest advocate would need to be assessed. PIAs would be appointed by the Prime Minister. ASIO would not be required to go through such a process, but simply get a warrant from the Attorney-General or, if they believe it is an emergency, the head of ASIO.

The “journalist information warrant” requirement is easily evaded, however — if the AFP wants to find out who leaked a government document, it can simply get the call data for all the public servants in the originating department without a warrant and check who called a journalist, rather than wasting time going through a warrant process.

Why should journalists be treated any differently to everyone else?

Australian law already recognises that for certain professions and groups — doctors and lawyers — confidentiality is so important that it must be acknowledged and respected by law. And the law also recognises that journalists have a right to protect confidences, under “shield laws”. Data retention has been found by a UK government investigation to have a “chilling effect” on investigative journalism. Less investigative journalism means the powerful and the corrupt are exposed to less scrutiny and have less to fear from exposure.

Putting aside that data retention is unjustified and bad policy anyway, the sort of protection that is being considered to be extended to journalists should also be extended to the medical and legal professions, and other professions that can make the case that confidentiality is crucial.

But this is about stopping terrorism — the ends justify the means.

Terrorism is a wildly overhyped threat in Western countries. About three times more Australians have died falling out of bed since 2001 than have died at the hands of terrorists; more Australians die from diseases like shingles and chickenpox than from terrorism. More women and children die at the hands of the partners and parents in Australia every year than the total number of Australian victims of terrorism in the last decade. More Americans die from causes like malnutrition, falls, swimming accidents and work accidents each year than the entire death toll from 9/11. The level of spending we direct toward national security is completely unjustified in terms of the harms it prevents.

Data retention would be yet another expensive, intrusive national security policy that has no objective justification. Doing things in the name of stopping terrorism relies on our emotional fear of attacks, rather than making the case for taking away our rights.

Finally – OK, I want to prevent my data from being collected – what can I do?

If you’re a criminal, or a terrorist, and government agencies want to spy on you, there’s not much you can do to stop them – thankfully. For the rest of us who are ordinary citizens engaged in lawful pursuits and not the special targets of concerted surveillance, there are a number of steps we can take to at least significantly reduce the amount of private information collected on us.

1. Be careful how you use social media. Don’t leave personal details or photos of yourself/your family on platforms like Facebook, for example, unless you have made the privacy settings as tight as possible. Intelligence agencies like ASIO actually employ “open source intelligence” firms who go through social media collecting data on people.

2. Use offshore services. Services like Gmail aren’t subject to Australian data retention laws. Companies like Google and Apple are also building in encryption as the default option in their products and services now, in response to the Snowden revelations and the criticism they have endured as a result. And take a lead from Malcolm Turnbull, one of the architects of our mass surveillance scheme: there are a growing number of ephemeral and encrypted messaging apps like Wickr (Malcolm’s favourite) that you can download which make it harder to monitor your communications. But go check the reviews of apps before you pick one — quite a few much-hyped services aren’t all they’re cracked up to be.

3. Use a VPN. There are lots of sites that now assess the merits of different Virtual Private Network providers in terms of security and privacy (in particular, whether they log what information is flowing through them), price and customer service. While there are free VPNs out there, $40-50 a year will get you a high-quality, fast VPN that will encrypt and anonymise your web traffic so that even your ISP doesn’t know what sites you’re visiting, and those sites only record the IP address of the VPN server you’re using, not your home address. And you can even install them on your smartphone. Needless to say, use an offshore VPN.

4. Use Tor. Tor is an easily-installed piece of software that bounces your traffic around the internet to disguise where it is coming from. Drawbacks are that you have to use it within a bespoke browser, and it can be very slow. On the plus side, you can choose to run your own Tor node and help provide anonymity to people in countries where there are serious consequences for saying the wrong thing online.

5. Use PGP or other encrypted email systems (but note, email content is not captured by data retention anyway). This can be a hassle and requires a bit more tech savvy. PGP is probably the best-known encryption program for email, and depending on who you ask, may or may not remain safe even from the NSA’s efforts to break it. But these aren’t “plug and play” apps — they require some work to install properly. Both Gmail and Yahoo’s free email systems will soon be end-to-end encrypted (even between the two systems), which might offer a more appealing solution for people wanting to protect email but lack tech knowledge.

6. Use smartphone voice/videocall apps. There’s no way to prevent the recording of your mobile phone data — your service provider has to know where your phone is and how you’re using it in order to provide you with basic telephony. But you can use IP-based communications apps that use the internet (via your VPN) rather than traditional telephone calls. And if you don’t want your location tracked, turn your phone off.

Let’s be clear: none of these offer perfect protection, by any means. But more importantly, they make the routine mass surveillance of the data retention regime significantly more difficult and therefore costly if it is going to be useful to agencies. In the absence of political will to rein in security agencies, the most effective means of halting their appetite for ever more surveillance is to drive up the cost of obtaining your private data. Mass adoption of these tools will do exactly that.