Australia's Mandatory Metadata Retention Legislation: Everything You Need To Know

The Government has officially introduced a Bill into the House Of Representatives today that would compel ISPs and telcos to store your data for a prescribed amount of time. This is everything you need to know about who will store your data, how long it will be kept for, and who can access it.

Update: Data Retention Bill Delayed Until 2015

Wait, What's Going On?

If you've been living on the Moon for the last six months, here's a quick backgrounder.

As part of a new anti-terrorism push, the Coalition Government led by Prime Minister Tony Abbott is looking to push through three pieces of new legislation.

The first Bill — which has already been passed into law — grants our law enforcement agencies more powers to hunt for bad guys; the second Bill is called the Foreign Fighters Bill and looks to establish "no go zones" for Australian travellers in a bid to stop locals leaving the country to train as lone-wolf terrorists; and the third is an amendment to the Telecommunications Interception Act which would compel ISPs and telcos to store customer metadata for the purposes of intelligence gathering and law enforcement for a period of two years.

The third bill for the TIA Amendment is the one being introduced into Parliament today, and that's the one that's most likely to upset Aussie geeks and privacy advocates.

So What's Being Stored, And For How Long?

Let's be clear: this is still just words on a piece of paper currently being argued over in Parliament. For a Bill to be passed into law, it first has to be read several times in the House of Representatives, before being passed up to the Senate or house of review. There it's debated over by, in this case, Senators who vehemently dislike the idea that Australian law enforcement agencies will be gathering up haystacks worth of metadata based on the threat that one day there might be a needle hidden inside of one.

Cross-bench Senators as well as Senators from the Australian Greens are opposed to the legislation, and have been for some time.

The Bill's passage through the House of Reps may also be slowed. Sources within the Opposition have told us this morning that the Labor Party will have to seriously consider the Bill before officially responding, rather than provide immediate bi-partisan support.

Either way, this is the last Parliamentary sitting day of the month, and the Parliament doesn't sit again until late November, so we have some time before things kick off in earnest once again in Canberra. Speaking at a press conference this morning, Attorney-General George Brandis wouldn’t be drawn on whether the Bill would be passed before the end of the year.

Here's what we know right now.

The Government has gone to pains to explain that it just wants your metadata captured by the new legislation, rather than what it calls "content".

Data like who called who when, which phone number was being used, basic customer account data like addresses and other relatively harmless data.

So where is the line being drawn?

Well, the government specifies that it doesn't want "session data" generated by customers. Here's an excerpt from the explanatory memorandum:

Under proposed paragraph 187A(4)(b), the retention obligation is explicitly expressed to exclude the retention of destination web address identifiers, such as destination internet Protocol (IP) addresses or uniform resource locators (URLs). This exception is intended to ensure that providers of internet access services are not required to engage in session logging, which may otherwise fall within the scope of the destination of a communication.

That means the government will be looking to store the date, time and duration of a communication (be it via web, phone or text), rather than the specific content of web browsing. That means the Government will know which IP addressed you browsed from and how long, but not the pages that you visited. Similarly, metadata would indicate who you called or texted and for how long it went for but not the content of the message.

Communications Minister Malcolm Turnbull said in a press conference this morning that the Bill isn't about garnering any "new" information, rather it's about creating a standard for data storage and duration of storage.

Law enforcement agencies can access content, but not under this legislation. For that, said agencies will require an explicit warrant. From the memorandum:

Accessing content, or the substance of a communication (for instance, the message written in an e-mail, the discussion between two parties to a phone call, the subject line of an e-mail or a private social media post), without the knowledge of the person making the communication is highly privacy intrusive and under the TIA Act can only occur under an interception or stored communications warrant, or in limited other circumstances such as in a life-threatening emergency. Interception is subject to significant limitations, oversight and reporting obligations. None of these arrangements are affected by this Bill.

Your metadata is going to be stored, as expected, for a period of two years. Storage regulations will be set up by the government to keep everything safe.

Basically, the government wants to store your data in order to smash individuals participating in the sharing of child exploitation material, planning criminal acts including terrorism and generating a trail of evidence in investigations to either rule in, rule out and successfully prosecute

Who Can Access Your Metadata?

During recent hearings relating to metadata access, concerns were expressed that tens of agencies were able to access metadata without a warrant in connection with various investigations.

In order to shore up access regulations and to cut down on abuse of warrantless access to metadata, the Government has specified which agencies will be allowed warrantless access to metadata stores, and which will need to actually apply for permission (emphasis added):

While telecommunications data is less privacy intrusive than content, law enforcement and national security agencies can only access data where a case can be made that this information is reasonably necessary to an investigation. This Bill will further strengthen privacy protections in the TIA Act in relation to data by limiting the types of enforcement agencies that can access telecommunications data.
Currently any authority or body that enforces a criminal law, a law imposing a pecuniary penalty or a law that protects the public revenue is an ‘enforcement agency’ under the TIA Act and can seek telecommunications data where that access complies with the requirements set out in Chapter 4 of the TIA Act. In 2012-2013 data was accessed by around 80 Commonwealth, State and Territory agencies with criminal law or revenue protection functions.
The Bill will require that bodies who are not a ‘criminal law enforcement agency’ for the purposes of the TIA Act must be declared by the Minister to be an ‘enforcement agency’ before they can authorise the disclosure of telecommunications data. These amendments will ensure that only authorities and bodies with a demonstrated need to have telecommunications information can authorise the disclosure of this information. These amendments are consistent with Recommendation 5 of the PJCIS Report that the number of agencies able to access telecommunications data be reduced.

Does Anyone Care About Your Privacy?

The Government will also appoint several safeguards to make sure that metadata access isn't being abused:

The Bill will further enhance privacy protections by introducing an independent oversight mechanism for access to data by law enforcement agencies. Under these provisions the Commonwealth Ombudsman will, for the first time, have the power to inspect the records of enforcement agencies to ensure that agencies are complying with their obligations under the TIA Act. The Inspector-General of Intelligence and Security (IGIS) currently oversights and will continue to oversight access to telecommunications data by the Australian Security Intelligence Organisation (ASIO).

Who Pays?

ISPs like iiNet have consistently warned that a massive cost would come with metadata storage. The government today has agreed with that assessment, but promised to contribute "substantially" to the creation and ongoing cost of the metadata retention program.

Turnbull said at this morning's press conference:

There are some ballpark figures being thrown around but they are at this stage not of sufficient accuracy for me to be citing. We will work through that will the PCJIS and of course we have a working group…and the Secretaries of the Attorney’s department and my department. We don’t have a final figure at this point. The estimates are getting more accurate but it’s something that will be refined in the course of the consultation.

Whether or not you'll pay more for metadata storage remains to be seen.

Is It Secure?

Various telcos, including Telstra, has expressed concern that the storage of large amounts of metadata at rest in a datacentre creates a "honeypot" for hackers. To address that concern, Malcolm Turnbull outlined plans to introduce new security regulations to protect Australia's telephone networks.

Speaking at a press conference this morning, Turnbull said that ultimately, storage is up to the telcos:

“Securing data safely is the responsibility of the telcos. They’re very alert to data security already. We are presently preparing new legislation which will strengthen…the security of Australia’s telecommunications structure or system, and that would add to that security and we expect those new laws and amendments to be in place before the 18-month implementation period is complete. [Security] is clearly in the hands of the telcos.
Most of the categories that we’re talking about in terms of data - telephone metadata — are being kept by phone companies…for up to 7 years. This is anticipating a change in that. There are some ISPs that have been storing data that they have hitherto kept for long periods down to short periods or at all. In an IP world, there isn’t a business need for them to do so. We’re asking them to do something which they’re either doing or supremely capable of doing, or which they may not in the future have a business need to do so. We’re talking about the degradation of this resource for law enforcement. There’s nothing new about…[agencies] accessing metadata.

Will It Ever Be Repealed?

So how long will you have to live with a data retention scheme? Is there any chance of the Government changing its mind?

Well, it's unlikely to be reversed if it actually gets passed through the House and the Senate, but the Parliamentary Joint Committee on Intelligence and Security — the one that originally examined the usefulness of a metadata retention program — will re-examine the program after three years.

Depending on when the program is introduced (keeping in mind there is the provision for an 18-month implementation lead time), the players on that panel, as well as the government itself, could be very different. That's very much a wait-and-see at this point, but it's unlikely to be repealed if it passes given that law enforcement agencies have been pushing so hard for it over the last two governments.

Got questions about Data Retention and the mandatory storage of your metadata? Let us know in the comments and we'll help you out!



    Will It Ever Be Repealed?
    So how long will you have to live with a data retention scheme? Is there any chance of the Government changing its mind?
    They have their collective foot in the door now... good luck getting it out..!

    Last edited 30/10/14 11:14 am

    I can see metadata being used to take someone down when there isn't enough evidence of another crime, similar to what happened with Al Capone and his tax evasion charges.

    The cat is almost out of the bag and the golden age of anonymity is almost over.

    Colour me surprised if, down the track a bit, it's being used exclusively by content owner "partners" to prosecute the evil bad content pirate terrorists. It's all part of the sell-out of Australian sovereignty to multi-national IP owners through the TPP.

      Well according to that they won't be able to prove anything?

      The monitoring is not allowed to capture IP's or URL's you have connected to. From what I can tell there isn't much in here that would be helpful for content criminals (Movie & TV makers).

        I thought I heard Turnbull himself say this morning it tracks user and destination addresses, just not content/URL.

        So they'll know your IP connected to [this ip] but not what specifically on that IP you accessed?

        Last edited 30/10/14 4:09 pm

    This would be a good time to have invested in hard drive manufacturers...

    As the SMH put it - "AFP commissioner Andrew Colvin said the scheme could 'absolutely' be used to tackle online copyright infringement [...] 'Illegal downloads, piracy, cyber crimes, cyber security, all these matters - our ability to investigate them is absolutely pinned to our ability to retrieve and use metadata,' he told reporters."

    So there you have it - tacit admission of what we knew all along - absolutely bubkis to do with 'terrorism' and everything to do with stopping people from pirating Game of Thrones.

    So they finally get around to defining what they want to store:
    - Your IP address, when you logged on and how long you were online
    - Nothing about what you did online - No IPs, no URLs, no data...
    - When you use a phone, who you called, when and how long.
    - No mention of cell tower pings, or logs.

    So, my question is; what does this achieve?
    Mr Terrorist Drug Dealer Pedophile turned on his modem at 11:30am 30/10/14, was assigned an IP, and remained logged in to that IP for the following 17 days, when his modem lost sync and then re-acquired a new IP.
    We have no idea what he did online for those 17 days...

    So - this proposed scheme does what exactly?
    Don't get me wrong, I thought the whole idea of blanket retention was a joke, where anyone who was hiding anything could just VPN/TOR/use someone elses Wifi, and as far as the ISP records are concerned there is no info...but...what does law enforcement get from knowing what IP address was assigned to Mr Terrorist Drug Dealer Pedophile or my 13 year old neice?

    I thought the argument was that law enforcement were worried that people were moving to VOIP, chat, etc and they were losing the ability to track those connections. I thought the "need" was that ISPs were not keeping the same kind of records that the cops were used to getting from Telcos - who called who, from where, when and for how long.

    As they say, the Devil is in the detail - but what does this achieve? Someone?

      My wife sees my Bank Statement meta data; a withdrawal happened in an area associated with gentlemen's clubs at 3am on Saturday morning. She cant see what I bought, but I reckon Id still be in the dog house.

        This law specifically states that Meta data stored is NOT allowed to contain the destination of access...
        In this law using your example:
        Bank statements reads 3AM. It doesn't even specify withdrawal, where or the amount.

      Yeah I am not sure really, but I really see this as a a foot in the door to be expanded by later governments. Governments often work with these kind of things to benefit 'government' whether it's theirs or a later government of the other party down the road.

      To me it seems to do a little now, because once this is there, expanding it will be much easier.

      Well it'd be something like this...
      Law enforcement raid [organisation] and confiscate their servers. Checking the logs (apache logs etc) they see that a certain list of IPs regularly visited the server, possibly at the same time.
      Law enforcement contact the ISPs associated with these IPs requesting the information about those IPs.
      Law enforcement find that that are regular matching sessions between the two account holders. Law enforcement then start investigating further links between those people.

        This sounds exactly like the sort of thing they are looking to target.

        As long as it is restricted to this I don't have a problem. We all know however that it A. won't be used legal and in this limited case of use or B. it will be changes soon after implementation so that they can do everything else.

      Well, at least in the case of piracy, one of the big things that's scuttled some cases abroad has been the fact that ISPs haven't kept a proper record of exactly which IP addresses were assigned to which people at a given time, so people have been falsely accused of things as a result.

      With just that information being kept, they could absolutely and without any room for doubt tie an IP address performing any nefarious deed online at a given time back to the account it is associated with and take action on the owner of that account. Obviously NAT etc means that you can't finger the exact device used, but you can certainly finger the owner of the account more accurately.

      Of course, what that means is that everyone with any remote level of technical know-how will just use a VPN or other service to cloak their IP and continue to plot their terrorist attacks and/or download Game of Thrones.

        The targeting of the account holder will not hold up in court fortunately. Because there a million ways that someones internet could be unknowingly used for illegal purposes. Eg. Unsecured wireless (or secured and breached), PC virus tunneling data for someone else, family member/house mate or ISP IP accounting error.

          Doesn't stop big media sending out infringement notices and pay-us-$2000-or-we'll-take-you-to-court blackmail letters though, unfortunately. :(

      they also monitor how much you download so if you have used all your 500gig allowance they will then assume you are a pirate and will hunt you down so local distributers can get money out of you for something that is owned overseas.......cos that is all it is about local distributors crying foul they cant monolpise movies and tv shows anymore or stop Australians legit paying for services like Netflix so they can watch stuff without being ripped off by the local distributors. its as simple as that. just like bricks and mortar stores complaing about steam or buying games from overseas cheaper. its all about the customer getting something at a fair price and cheaper than the ripping off of Australians by their own local distributors.... seems fair doesn't it.

    Let's see if I got this right. Using a secure, non-Australian VPN Service will do a pretty good job of keeping your internet activity safe from spying?


      Let's develop a system that will cost the Australian public 100s of millions of dollars to establish, manage and maintain. Only to catch a handful of bottom-feeding, computer-illiterate criminals.

        well said, seems good value for money doesn't it. but as long as the local distributors get paid they will all say it was worth it.....

    There has a been a bunch of talk this morning around email metadata being captured. How does this work if you use a service like Gmail/Hotmail etc? Being international businesses do the laws apply to them as well?

      Any international company doing legitimate business in Australia has to abide by Australian laws. So yes, they'd be bound to comply.

      Thanks to Edward Snowden we know that the NSA capture this information already and pass it on when requested to other members of the five eyes.

    Just a thought, but if all Australian IP traffic is going to be stored, isn't the load of every DDOS under-way across the country going to pretty much crash the system daily?

    I put my metadata down the S bend when I'm finished.

    It'll go something like this:
    ASIO man 1: Mr Bad Man starts up router, gets IP address
    ASIO man 2: Damn, we don't know what he did with that connection.
    ASIO man 1: That's ok, call our partners overseas *cough* NSA *cough* ... they'll have all the data we need...

    Does this mean that the police can access my phone records to tell I made a phone call while driving and prosecute me for doing so?

    good luck linking an IP address with an individual user.

    Time to make a device that random searches different destinations while I'm surfing the internet with the same MAC address as my machine! If we each have a device that create's enough crap internet traffic the idea will become economically unfeasible because:- 1 they will have to store the traffic data and 2 they will have to sort the crap from the real stuff. I refuse to use a VPN, because I shouldn't have to. I wonder how they would fair legal trying to prove it was me and not an automated device accessing different things! Basically, what I am saying is we create a huge amount of spam!

      App starts on PC startup.
      Reads from a dictionary of terms to send random HTTP requests.
      Sends one request per second (with a different query) to Google, Yahoo, Amazon, eBay and Bing.
      Delay requests and use multiple servers to not effectively DDOS a company.
      App will remind the user every week about it's existence and to check whether it should be disabled/removed.

      How's that grab you?

    So, I can't even access my own metadata they have stored because I'm not a criminal law enforcement agency?

      Several defence attorneys in the US have begun requesting metadata information from the NSA, particularly mobile cell tower locations in order to prove their clients were not in the area where the alleged crime occurred. So far the NSA has refused to release that information. Would defendants be allowed to access this information in criminal cases in Australia? Would plaintiffs be able to access the information in a civil case in Australia?

    You're damn right the ALP has to seriously consider it. Their recent support of anti terror legislation has only seen the Greens vote increase.

Join the discussion!

Trending Stories Right Now