The Colossal Costs of Building UK's Monster Surveillance Network

The UK will have to build a mammoth network of Internet surveillance centers if the government passes its Investigatory Powers Bill - dubbed the the Snoopers' Charter - into law.

The proposal, which the Home Office wants to rush through the House of Commons just after Easter, will cost the country billions of pounds. The centers will be required to keep large databases of all the connections made by UK Internet users for one year — and to share them automatically with the UK's government and intelligence agencies.

Miss the #EU-#US #PrivacyShield hearing w/ @maxschrems @EP_Justice yesterday? Watch the video recording & read more: https://t.co/2JiJnmcSbr

— LIBE Committee Press (@EP_Justice) March 18, 2016​

The government is bracing itself for the vote as the news arrives that the only other country in the world to have ever tried a similar approach — Denmark — has just decided to abandon the plan, for the second time in ten years.

The first Danish "session logging" system was put into place in 2007, but was abandoned in 2013 after the country's police and security services found it to be practically useless — besides being very expensive for Internet providers to install and operate.

@dcampbell_iptv @HelpSnowden UK has always spent huge amounts on war machinery alone, so intel cost unlikely to deter? Plus cost is secret

— Kristian Harstad (@KristianHarstad) March 21, 2016​

Another attempt to build an improved system, carried out by the Danish Ministry of Justice at the start of March 2016 also appears to have foundered. 

Before the final decision was taken, the Danish government asked accounting firm Ernst & Young to ascertain how much the new surveillance network would cost.

The experts found that total expenses would be around one billion Danish Krone (US$150 million). The Danish government decided that the costs were too high for the country and its tech sector.

In the UK, the costs are likely to be much-much higher. If in Denmark — a country of 5.6 million people — the government estimated that each citizen would produce about 62,000 records every year, in Britain, whose population is about ten times the size of Denmark's, the final annual database would have to include about four trillion a year.

Other estimates suggest that the sheer amount of records could even hit tens of trillions every year. That is because each of those records, as per the law, would have to contain: a customer account reference or device identifier; the date and time of the event; the duration; the source and destination IP and port number of each session; the domain name or linked URL; the volume of data; and the name of Internet service you connected to.

The UK will have to find a way to store an enormous amount of information every day — even if each record's weight was brought down to 100 bytes, on a yearly level, we are talking exabytes (thousands of petabytes). 

The only surefire way to deal with this information is by building new massive data centers, which will need at least US$140 million in equipment to handle each exabyte. Add the building, as well as cooling and electricity management and you have only started understanding the eventual costs of the UK's new monster surveillance plans.