Surveillance State

How popular can a piece of software get before being in "beta" is no longer a legitimate excuse for known software flaws? Or, to put it another way, is it responsible to allow hundreds of thousands of people to install your product, when you know ahead of time that doing so opens them up to attack?

The software visionaries at the Mozilla Corporation, which makes the popular Firefox web browser, have taken the approach that creativity and functionality is king--even if security has to take a backseat. Case in point: The widely praised "Ubiquity" software add-on, which brings an amazingly rich and extensible new form of interaction to the Firefox Web browser.

The technology press has showered praise upon the developers of this software tool. However, in prioritizing functionality over security, Mozilla Labs punted complex trust choices to end users--the vast majority of whom are ill-equipped to make such decisions. The end result is that the hundreds of thousands of users of Ubiquity face a significant risk of browser hijacking by attackers, which could result in the theft of e-mail and online banking account information.

Mozilla's Ubiquity in Action

Updated:This post originally contained incorrect information about Sentinel's products. That has been corrected (see below).

Attorneys general from a number of states have given their support to a collection of weak and ineffective age verification technologies, all of which aim to protect children on the Internet. At a meeting of the Internet Safety Technical Task Force at Harvard University on Tuesday, the consensus seemed to be that while none of the technologies actually work, doing anything at all was better than nothing. Simply put, no one wants to be blamed for inaction against online child predators.

Kicking off the meeting, Richard Blumenthal, the Connecticut attorney general, summed up the general expectation of the other 48 state attorneys general involved in the effort: "If we can put a man on the moon, we can make the Internet safe (for children)." Unfortunately, while the federal government sunk billions of R&D dollars into NASA's space efforts, the AGs have yet to cough up any research funds, and seem to expect industry to come up with their own solutions.

Won't someone think of the children?
Given the intense political pressure to do something about child safety online, and a complete lack of proven, peer-reviewed, and abuse-resistant technologies available on the market, a number of private companies have stepped in to fill the void--with products that can at best be described as ineffective, and at worst as snake oil.

Several age verification solutions were presented at the task force meeting, from companies that included Aristotle, IDology and Sentry. All of the companies seem to do pretty much the same thing--collecting information from public records, and then prompting users to enter some of this info when they wish to log in to an "age restricted" Web site. One example of this is the rated R movie trailers of many Hollywood movie studios, which require a user to enter in his or her name, ZIP, and date of birth before playing the trailer.

This form of verification has been repeatedly criticized as "laughable" by security experts. As a test, I was able to successfully view the trailer for Sony's new thriller movie, Quarantine, by giving the name, date of birth, and ZIP code of vice presidential candidate Sarah Palin, all of which were available on the politician's Wikipedia page. Sony Pictures uses an age verification service from Sentinel (another company which presented at the task force meeting), which seems to only protect the fragile eyeballs of technologically unsavvy youngsters who have not yet learned how to use a search engine.

During the question and answer sessions following their presentations, each of the age verification and other child safety technology vendors admitted that their products are neither bullet proof nor even that difficult to evade. However, they all generally preached a belief in the security benefits of "raising the bar" and providing a "bump in the road."

Speak softly and carry a big stick
With companies and politicians falling over themselves to prove how much they are doing to keep children safe, it is worth taking a look at the incentives and motivations of this industry.

First, the politicians: Attorneys general from 49 states have been focusing on this issue for some time, culminating in an agreement signed with MySpace back in February of this year--the only state to reject the deal was Texas, whose AG felt that the deal didn't go far enough. This is an issue that carries a lot of weight with voters, and as New York AG Andrew Cuomo's recent strong-arming of ISPs over their Usenet news feeds has demonstrated, easy political wins can be gained with little to no pushback from the tech industry.

Second, the social-networking sites: Facebook and MySpace, the 500-pound gorillas of the industry don't seem to be too keen to adopt any of the existing solutions pitched by vendors--primarily because the technology doesn't do much, won't stop abuse, and will cost the companies money. While News Corp's MySpace certainly has deep pockets and could easily pay a couple million for age verification software, the company appears to be resisting calls to do so primarily out of an urge to avoid a slippery slope. That is, if the social-networking site can be pressured into forcing its user base to jump through one level of inconvenient and burdensome verification, other demands will soon follow.

Third, the "solution" vendors: This collection of companies rely upon fear to sell their products--not so much fear of the abuse of children by predators, but the fears of companies and politicians that they will be accused of not doing anything. These firms are not selling complete solutions to the problem of age verification (since one does not exist)--but are selling excuses. That is, if social-networking sites purchase their products, and children are later groomed or abused online, the companies will at least be able to claim that "we've purchased and used the best age verification products that industry offers. Don't blame us--we've at least tried to do something."

The not so thinly veiled threat aired at the event was that if the industry didn't police itself, the various state AGs might have to push for regulation. The fact that the technology isn't effective doesn't seem to be a major cause for concern. All that really seems to matter, at least for the policy makers, is that the industry do something, which can then be sold to voters back home as a success in protecting little Jane or Johnny.

The offshore problem
The elephant in the room in this debate is the issue of foreign Internet companies. That is, if American social-networking sites are forced to implement oppressive and burdensome age verification rules, teens may ditch MySpace and head to a Chinese, Brazilian, or Indian Web company, where a user's age is not verified.

Internet users are a fickle bunch--that is, they are not particularly loyal to brands, and if a company's product ceases to be cool, users will leave in droves. As an example, just look to Friendster, which was at one point the most popular social-networking site on the Internet. Once MySpace offered a better, more enjoyable experience, Friendster turned into a cyber-ghost town. While the network effect is indeed a powerful and sticky force, a lame user experience will be more than enough to make users leave for greener pastures.

Now, as another example, consider the case of Napster, the first peer-to-peer file-sharing company. Remember that for a time, Napster was the most popular file-sharing tool on the Internet, with tens of millions of users. As an American company, once Congress got wind of the file-sharing phenomenon, it was able to hold hearings, and force the CEO of Napster to appear before the Senate Judiciary Committee.

Fast forward a couple years: Napster had been sued into financial oblivion, and America's teens had moved on to a significantly more legislation-resistant file-sharing platform--Kazaa. This file-sharing company, designed by three men from Sweden, developed by programmers in Estonia, headquartered in Australia, and incorporated in the south pacific island nation of Vanuatu, was global in scale, and for the most part, completely beyond the reach of America's laws.

Whatever you think of file-sharing, there is one thing that is beyond debate: Due to a change in the legal environment, Americans abandoned, en-masse, an American company's P2P offerings, and instead signed up for the services offered by a foreign company whose CEO could never be hauled before the U.S. Congress. Furthermore, while Napster was primarily a service offering free music downloads, the Kazaa platform offered easy access to music, movies, pirated software, and pornography (of both legal and illegal varieties)--all from the same easy to use graphical interface. That is, by chasing file-sharing underground, we completely gave up any possibility of lightly regulating it.

No one present at Tuesday's Task Force meeting had any solutions to this problem, nor were they too keen to discuss it. It would be cruelly ironic if in an effort to protect America's youth online, those same children were chased into the hands of unscrupulous foreign firms with little incentive to protect their users from predators and other forms of harm.

Update: The original version of this blog post included Sentinel in the list of companies who push weak age verification software to social networks. In fact, Sentinel has voluntary withdrawn its age verification products from the social networking market, although it continues to supply the easy-to-evade product to Hollywood movie studios.

Disclosure: I am a paid student fellow at the Berkman Center at Harvard University, which participates in and hosted the meeting of the Internet Safety Technical Task Force. In particular, professor John Palfrey, the chair of the Task Force, is also the Faculty co-director of the Berkman Center, where I work. I have neither consulted with Palfrey, nor any of my other colleagues at Harvard with regard to this blog post. It reflects my own opinions, and certainly not those of Harvard or any of the other people associated with the Berkman Center.

Now that the FCC has delivered a smackdown to Comcast for its sketchy anti-BitTorrent activities, it's about time that some other company stepped up to the plate and breathed life into the Net neutrality debate. Surveillance State is happy to report that the Walt Disney-owned ESPN sports network, through its selective blocking of people from particular Internet service providers, may very well wake the sleeping giant that is Net neutrality.

ESPN360.com bills itself as the premier destination for streaming access to live sports events. If the sport or team you love isn't important enough to be shown on cable TV, no fear, ESPN will stream it to you online for free. Well, that is if you a subscriber to the right Internet service provider.

ESPN's warm welcome to customers of ISPs that have signed deals.

(Credit: ESPN360)

Customers of AT&T DSL and Verizon's Fios services, along with approximately 20 more ISPs, can have free, 24-hour per day access to ESPN's exclusive sports content. Customers of Comcast, Cox, and hundreds of other ISPs, both big and small, are left out in the cold--forbidden to access content that ESPN has, via exclusive contracts, guaranteed that you cannot obtain via any other means in the U.S.

Love Italian soccer and get your Internet access through Comcast? Too bad.

After telling out-of-luck users that their ISPs haven't coughed up funds for their customers to access ESPN360, the sports network informs them that AT&T customers do have access, and helpfully provides them with a toll-free number that they can call to make the switch to that ISP. How nice of ESPN.

ESPN's message to Comcast's customers

(Credit: ESPN360.com)

There are many reasons why an ISP would decide against paying ESPN for its premium Web content. A spokesperson for Cox Communications told a journalist back in 2006 that signing on to carry ESPN360 would require Cox to burden all of its customers with additional costs--even those who don't want the service.

Many customers in the United States still have no real choice for their ISP. For example, if you live out in rural Montana and the one cash-starved regional ISP that offers broadband Internet access hasn't agreed to ESPN's shakedown effort, you have no options.

Not surprisingly, this discriminatory policy concerns Ben Scott, policy director at Free Press, the group leading the fight against Comcast's anti-BitTorrent filtering and other foes of Net neutrality. When asked for his view, he issued the following statement:

ESPN360 raises the unsettling prospect that each ISP will someday have its own distinctive "Internet experience" that includes all kinds of exclusive content in parallel walled-gardens. That is a troubling vision for anyone that values an open media system shared by all Internet users alike.

Most interesting, I suppose, is ESPN's policy of discriminating against particular ISPs, while at the same time giving free access to any user visiting the site from a U.S. military or university Internet connection; that is, users coming in via a .edu or .mil IP address get to view the sports content without any money changing hands between ESPN and Uncle Sam.

While the decision to support the troops (via free access to European soccer) is a noble one, the decision to give college students a free ride is extremely interesting. After all, the major media companies have shown no real restraint in trying to shake down university users--at times, taking thousands of them to court for their attempts to download content for free.

The cynical among us might perhaps see this as a Joe Camel-esque tactic--offer free access when they're young, hope that they develop a habit, and once they graduate or leave the military, they'll look for an ISP that has cut a deal with ESPN.

ESPN spokesman Paul Melvin dismissed my cynicism, explaining the decision to offer free service to these millions of Americans:

These groups are not commercially served by an ISP, and they are not likely to be commercially served in the reasonably foreseeable future. Given this, there is no reasonable chance that we could strike a deal with a retail ISP, nor that the market will continue to grow and offer them greater choice. As a result we adjusted to these specific circumstances.

To try to understand how government regulators would see this issue, I turned to Rep. Rick Boucher (D-Va.), one of the most powerful members of Congress in issues related to telecommunications and Net neutrality.

[This issue has] nothing to do with network neutrality debates, which focus on the practices of the broadband providers. What is in question, is the practice of a content provider, a website owner, in terms of how it chooses to make its content available ... I don't see it as a matter for policy makers to get involved in. I see it as a matter for private contracts, to be determined by content providers.

The congressman is correct in that this is not a traditional Net neutrality conversation per se, since that term usually applies to discrimination by the company owning the "last mile"--the connection to a user's home. Perhaps a new term will need to be invented by the "Save the Internet" crowd, so as not to further dilute the "Net neutrality" phrase. However, what does concern me is the rather shameless attempt by ESPN to shake down big ISPs, while at the same time giving away its content to millions of college students for free.

Boucher added that:

If ESPN had market power, i would agree that there would be anti-trust issues. Companies that have market power have different market obligations. [However], this is one web site that is putting up sports content, competing with others. Even though ESPN is popular, I don't think [anti trust] applies. It might in TV broadcast, but certainly not on the Internet.

While I respect the congressman (and am a huge fan of his work in fighting against the dreaded Digital Millennium Copyright Act), I think he is on the wrong side of this issue. Due to the exclusive contracts that ESPN has negotiated with various sports associations, the company does have market power. If you love European soccer or another sport that can't draw enough viewers to justify TV coverage, there is simply no other (legal) way to view live sports events in the U.S. ESPN is the only game in town.

Libertarians out there will, like the congressman, argue that ESPN is a private company and has a right to decide which customers can access its content. If ESPN offered a generic service (like e-mail, horoscopes, or photo sharing), that would certainly be true. However, because ESPN has exclusive contracts for U.S. distribution of many types of sports content, I don't think these same rules apply. ESPN shouldn't be able to get exclusive access to this content, and then deny it to millions of Americans.

Yes, the content is expensive--which is why ESPN could allow the customers of non-kickback-paying ISPs to pull out their credit cards, and sign up for an individual account in order to view these games. Unfortunately, this is not something ESPN is interested in. Explaining this lack of an individual subscriber option, ESPN's Paul Melvin simply stated that "it is not the business model that we've chosen."

Google announced on Monday that the company will be reducing the amount of time that it will keep sensitive, identifying log data on its search engine customers. To the naive reader, the announcement seems like a clear win for privacy. However, with a bit of careful analysis, it's possible to see that this is little more than snake oil, designed to look good for the newspapers, without delivering real benefits to end users.

In a post to the company blog on Monday, the company announced that it will be significantly reducing the amount of time that it hangs onto identifying user data in its Web server logs:

Today, we're announcing a new logs retention policy: we'll anonymize IP addresses on our server logs after 9 months. We're significantly shortening our previous 18-month retention policy to address regulatory concerns and to take another step to improve privacy for our users.

Hidden further down in the blog post, were a few more details:

We haven't sorted out all of the implementation details, and we may not be able to use precisely the same methods for anonymizing as we do after 18 months, but we are committed to making it work.

Google's announcement was extremely light on details, specifically, how the company planned to anonymize the records after 9 months. I contacted Google to find out more, and received an extremely interesting reply:

After nine months, we will change some of the bits in the IP address in the logs; after 18 months we remove the last eight bits in the IP address and change the cookie information. We're still developing the precise technical methods and approach to this, but we believe these changes will be a significant addition to protecting user privacy.... It is difficult to guarantee complete anonymization, but we believe these changes will make it very unlikely users could be identified.... We hope to be able to add the 9-month anonymization process to our existing 18-month process by early 2009, or even earlier.

To understand what this means (and how useless the new privacy "enhancements" are), consider the following:

When a user conducts a search using Google's search engine, the company stores three main types of information in a log file: the user's IP address (which is a unique network address given to her computer by her Internet service provider), the words that she searched for, and her cookie identifier (a unique value given to every Web-browser that visits a Google Web-property).

As per Google's existing policy, after 18 months Google "anonymizes" the IP address and cookie information from its logfiles. While the company hasn't said how it de-identifies the cookies, it has revealed in public statements that its IP anonymization technique consists of chopping off the last 8 bits of a user's IP address.

As an example, an IP address of a home user could be 173.192.103.121. After 18 months, Google chops this down to 173.192.103.XXX.

Since each octet (the numbers between each period of an IP) can contain values from 1-255, Google's anonymization technique allows a user, at most, to hide among 254 other computers. In comparison, Microsoft deletes the cookies, the full IP address and any other identifiable user information from its search logs after 18 months.

Google has now revealed that it will change "some" of the bits of the IP address after 9 months, but less than the eight bits that it masks after the full 18 months. Thus, instead of Google's customers being able to hide among 254 other Internet users, perhaps they'll be able to hide among 64, or 127 other possible IP addresses.

By itself, this is a laughable level of anonymity. However, it gets worse.

First, remember that Google will not delete or anonymize user cookies from the logs when it slightly smudges IP addresses after nine months. Second, remember that as long as you use a Google Web property at least once every two years, the company will maintain a unique identifiable cookie value within your Web browser.

Thus, consider the following scenario:

In June 2008, a user from 173.192.103.121 with cookie value 12345 conducts a search for "breast cancer risks." Nine months later, in March 2009, the company scrubs some portion of the IP address, perhaps to 173.192.103.1XX. However, the cookie remains in the log.

In April 2009, that same user returns to Google, and conducts a search for "stephen colbert youtube videos," again from the same IP and the same cookie value 12345.

Even though the 9-month-old search logs have been "anonymized", because the cookie values remain, it is trivial to match the newer search results to the older searches, and thus completely reverse the anonymization process.

The simple truth is that any IP anonymization technique, no matter how strong or weak, is simply a waste of time, if cookie values are not also anonymized.

Unfortunately, Google is relying on the fact that the mainstream media (I'm looking at you New York Times and Washington Post) are clueless on these issues, as well as seemingly most of the technology press. Google's new anonymization policy is totally worthless, and the company deserves to be called out for its deception.

Disclaimer: I interned at Google during the summer of 2006 and received a $5,000 Google fellowship in both 2006 and 2007. I have also interned or worked for both the Electronic Privacy Information Center (EPIC) and the American Civil Liberties Union (ACLU) of Northern California, public interest groups that have been extremely critical of Google's privacy policies.

If you thought that the National Security Agency's warrantless wiretapping was limited to AT&T, Verizon and Sprint, think again.

While these household names of the telecom industry almost certainly helped the government to illegally snoop on their customers, statements by a number of legal experts suggest that collaboration with the NSA may run far deeper into the wireless phone industry. With over 3,000 wireless companies operating in the United States, the majority of industry-aided snooping likely occurs under the radar, with the dirty-work being handled by companies that most consumers have never heard of.

A recent article in the London Review of Books revealed that a number of private companies now sell off-the-shelf data-mining solutions to government spies interested in analyzing mobile-phone calling records and real-time location information. These companies include ThorpeGlen, VASTech, Kommlabs, and Aqsacom--all of which sell "passive probing" data-mining services to governments around the world.

ThorpeGlen, a U.K.-based firm, offers intelligence analysts a graphical interface to the company's mobile-phone location and call-record data-mining software. Want to determine a suspect's "community of interest"? Easy. Want to learn if a single person is swapping SIM cards or throwing away phones (yet still hanging out in the same physical location)? No problem.

In a Web demo (PDF) (mirrored here) to potential customers back in May, ThorpeGlen's vice president of global sales showed off the company's tools by mining a dataset of a single week's worth of call data from 50 million users in Indonesia, which it has crunched in order to try and discover small anti-social groups that only call each other.

Slide from "Identification of Nomadic Targets " ISS Webinar

(Credit: ThorpeGlen)

Clearly, this is creepy, yet highly lucrative, stuff. The fact that human-rights abusing governments in the Middle East and Asia have deployed these technologies is not particularly surprising. However, what about our own human-rights-abusing government here in the U.S.? Could it be using the same data-mining tools?

To get a few answers, I turned to Albert Gidari, a lawyer and partner at Perkins Coie in Seattle who frequently represents the wireless industry in issues related to location information and data privacy.

When asked if there is a market for these kinds of surveillance data-mining tools in the U.S., Gidari told me: "Of course. It is a global market and these companies have partners in the U.S. or competitors."

The question is not if the government would like to use these tools--after all, what spy wouldn't want to have point-and-click real-time access to the location information on millions of Americans? The real mystery is how the heck the National Security Agency can legally get access to such large datasets of real-time location information and calling records. The answer to that, Gidari said, is the thousands of other, lesser-known companies in the wireless phone and communications industry.

The massive collection of customer data comes down to the interplay of two specific issues: First, thousands of companies play small, niche support roles in the wireless phone industry, and as such these firms learn quite a bit about the calling habits of millions of U.S. citizens. Second, the laws relating to information sharing and wiretapping specifically regulate companies that provide services to the general public (such as AT&T and Verizon), but they do not cover the firms that provide services to the major carriers or connect communications companies to one other.

Thus, while it may be impossible for the NSA to legally obtain large-scale, real-time customer location information from Verizon, the spooks at Fort Meade can simply go to the company that owns and operates the wireless towers that Verizon uses for its network and get accurate information on anyone using those towers--or go to other entities connecting the wireless network to the landline network. The wiretapping laws, at least in this situation, simply don't apply.

Giardi explained it as follows:

Networks are more and more disaggregated and outsourced, from customer service call centers overseas with full viewing access to data to key infrastructure components and processing. A single communication is handled by many more parties than the named provider today. Moreover, interoperability protocols include network identifiers--send a message from company A to company B and the acknowledgment of delivery may include location and other information. That's just the way the system is designed--location was about billing in the early years and no one bothered to undo the existing protocols when business models changed and interoperability became common practice or a myriad of new messaging companies came into being...So my point is that there are many access points--albeit less convenient than one-stop shopping at the big carriers--to get information including real-time data.

ThorpeGlen's product appears to be a mashup of Google Earth + phone location data (in this case, from 50 million people in Indonesia)

(Credit: ThorpeGlen)

For example, if a Sprint Wireless customer in Virginia calls a relative in Montana--who is a customer of a small, regional landline carrier--information on the callers will spread far beyond just those two communications companies.

Sprint doesn't own any of its own cellular towers, and so TowerCo, the company that owns and operates the towers, of course, learns some information on every mobile phone that communicates with one of its towers. This is just the tip of the iceberg, though. There are companies that provide "backhaul" connections between towers and the carriers, providers of sophisticated billing services, outsourced customer-service centers, as well as Interexchange Carriers, which help to route calls from one phone company to another. All of these companies play a role in the wireless industry, have access to significant amounts of sensitive customer information, which of course, can be obtained (politely, or with a court order) by the government.

With the passage of laws like the FISA Amendments Act and the USA Patriot Act, in most cases, requests for customer information come with a gag order, forbidding the companies from notifying the public, or the end users whose calling information is being snooped upon. Gidari summed it up this way:

So any entity--from tower provider, to a third-party spam filter, to WAP gateway operator to billing to call center customer service--can get legal process and be compelled to assist in silence. They likely don't volunteer because of reputation and contractual obligations, but they won't resist either.

Seeking clarification, I turned to Paul Ohm, a former federal prosecutor turned cyberlaw professor at the University of Colorado Law School and a noted expert on surveillance laws.

Before getting into the details of the issue, Ohm first outlined the basic problem of the various wiretap and surveillance laws; they are extremely confusing and few people fully understand them. The 9th Circuit Court of Appeals seemed to share Ohm's view, stating a few years ago that the Electronic Communications Privacy Act is a "complex, often convoluted area of the law" (United States v. Smith, 155 F.3d 1051).

Ohm then said that the "one thing I can say with confidence is that you are correct to note that the [Stored Communication Act's] voluntary disclosure prohibitions (in 18 USC 2702(a)) apply only to providers to the public."

After describing all the ways that the government could legally collect real-time data on millions of U.S. citizens, Gidari said that essentially, the existence of such a program would likely remain a secret (barring a whistle-blower or leaks to the press by government officials). Summing it up, he stated that:

Whether [a] vendor to a carrier to the public cooperates with agencies (either for a fee or by acquiescence in an order), is something you will not find out as FISA makes it so, regardless of whether the person is in the U.S. or communicating with a person abroad. Such means and methods largely are hidden.

However, if the existence of such a program were ever confirmed, Ohm said that Congress would not be too happy:

If [the sharing of data by niche telecom providers] is seen as allowing an end-around an otherwise clear prohibition in the SCA, Congress is likely to throw a fit when it is revealed and try to amend the law. DOJ is sensitive to this kind of thing (despite what the NSA wiretapping program would lead you to believe) and would probably try to avoid blatantly bypassing otherwise clear language in this way.

An internal review by University of Colorado officials has found that a controversial research project conducted by a team of computer scientists did not constitute research misconduct. University lawyers have also stated their belief that the team probably did not violate US wiretapping laws.

As I reported in a blog post yesterday, a team of researchers from both the University of Colorado and University of Washington recently presented a controversial study in which they recorded a limited portion of the communications of users of Tor -- a popular anonymizing proxy network.

According to a written statement posted by the research team, an internal university review conducted on the 24th of July 2008 found that:

Based on our assessment and understanding of the issues involved in your work, our opinion was that by any reasonable standard, the work in question was not classifiable as human subject research, nor did it involve the collection of personally identifying information. While the underlying issues are certainly interesting and complex, our opinion is that in this case, no rules were violated by your not having subjected your proposed work to prior IRG scrutiny. Our analysis was confined to this IRG (HRC) issue.

In a statement made to the Boulder Daily Camera newspaper today, spokesman Bronson Hilliard said that University attorneys described the wiretap law as "broad." He added that "legal counsel's opinion was that there's no clear indication that there was any kind of criminal action on the part of the researchers."

The Electronic Communications Privacy Act (ECPA), which governs network surveillance and access to private stored communications is particularly difficult to understand, something the US 9th Circuit Court of Appeals recognized when it described ECPA as "a complex, often convoluted, area of the law" (pdf). Computer scientists simply have no business making judgments about the legality of network monitoring and interception research -- and should, as the EFF advises, seek legal advice before doing so.

While I have strong personal objections to the methods employed by the researchers, the primary criticism in my original blog post was that the researchers had not sought a review of their project by university lawyers and the school's human subjects review board before conducting their study. Given that the University of Colorado was able to conduct both of these within 12 hours of the publication of my blog post yesterday, it is difficult to see how seeking such reviews ahead of time would have been any significant burden.

Personally Identifying Information

In reaching its decision, the University of Colorado review determined that the researchers did not collect any "personally identifying information" from users of the Tor network. This is in spite of the fact that for 15 days, the researchers collected the unique network addresses of each user sending data through their server.

While that may be the view of the University, there are certainly others that disagree. Back in February of this year, the European Union announced that it now considers IP addresses to be personally identifiable information.

IP addresses have been used by law enforcement to justify FBI raids on homes, by the record companies in copyright infringment suits, as well as in foreign countries, where suspects have been arrested and beaten because their IP addresses appeared in an incriminating log files.

In the last few weeks, there has been a significant amount of discussion of this issue, after a court ordered YouTube to hand over the IP addresses of millions of users to Viacom as part of its massive copyright infringement suit against the video sharing site. While Google (which own YouTube) has long argued that IP addresses are not personally identifying information, at least with regard to calls for the company to delete its own search log files, it rapidly changed its position once it was faced with the possibility of handing such data over to Viacom.

"Safe" storage of data

The researchers themselves admit that the data that they have collected is extremely sensitive. In their statement issued yesterday, they stated that "we took extreme caution in managing these traces and have not and will not plan to share them with other researchers."

If the information was not sensitive and could be potentially used to identify Tor users, why would they need to take such care managing the data, and why could they not share it with others? If it is not personally identifying information, why don't they put it online?

The fact is that this information is extremely sensitive, and were it to fall into the wrong hands -- an oppressive foreign government that does not take kindly to anonymous speech -- users whose IP addresses could reveal their identity could soon find themselves subject to arrest, imprisonment or torture.

While we can be asked to trust this research team not to share the data with others, there is little that they can do if presented with a government subpoena, or other lawful request. Furthermore, there is always the risk that they could accidentally lose the data, or be the victim of data theft.

Finally, the researchers have not said how long they plan to hang onto this data. As much as I criticize Google, at least they partially anonymize their server logs after 18 months.

The only safe and responsible way to handle this sensitive data is to delete it. Anything else is simply irresponsible..

Be Nice to Privacy

To be clear -- my focus on this issue is not about enforcing the law, no matter how flawed it may be. There are many unjust laws that I despise, chief among them the Digital Millennium Copyright Act, and I will eagerly defend researchers who violate these.

Communications privacy laws, unlike the DMCA, are (mostly) written for our protection. After spending the last several months criticizing AT&T, and later the US Congress' complete capitulation for illegal wiretapping immunity, I do not see how I could rightfully defend these researchers. Yes, they had good intentions -- but then, so might have the Bush Administration when it asked the telecoms to help it spy on millions of Americans.

A group of researchers from the University of Colorado and University of Washington could face both civil and criminal penalties for a research project in which they snooped on users of the Tor anonymous proxy network. Should federal prosecutors take interest in the project, the researchers could also face up to 5 years in jail for violating the Wiretap Act.

The team of two graduate students and three professors neither sought legal review of the project, nor ran it past the Human Subjects Committee at their university, putting them in a particularly dangerous position.

The academic paper, "Shining Light in Dark Places: Understanding the Tor Network" (pdf) was presented at the Privacy Enhancing Technologies Symposium yesterday, in Leuven, Belgium. The authors are listed as: Damon McCoy, Kevin Bauer, Dr. Dirk Grunwald, Dr. Tadayoshi Kohno and Dr. Douglas Sicker.

The goal of the project was to learn what kind of traffic was flowing over Tor -- a free network providing anonymous web and other Internet services to hundreds of thousands of users world-wide. Some of Tor's users include pro-democracy dissidents, journalists and bloggers in countries like China, Egypt and Burma who would otherwise face arrest and torture for their work.

Tor relies on volunteers who donate computing power and bandwidth to run approximately 2500 publicly accessible proxy servers, which are then used by hundreds of thousands of people to hide their Internet traffic.

In order to study Tor, the researchers setup their own 'exit node' server on the University of Colorado's high-speed network. For 4 days in December 2007, they logged and stored the first 150 bytes of each network packet that crossed their network, thus revealing what kind of traffic was crossing the network, and the remote websites that Tor users were visiting. While the authors do not state how many sessions they snooped on, they do state that their server carried over 700GB of data.

In a second part of the study, the researchers ran an 'entry node' to the network for 15 days, which allowed them to determine the source IP address of a large number of Tor users. They used this to learn which countries use Tor more heavily than others. Note that in this second part of the study, the researchers did not have access to the destination site information, nor were they able to observe the kinds of traffic going through their server.

The researchers found that HTTP (web traffic) was responsible for 58% of their servers' bandwidth. They also found that the BitTorrent file-sharing protocol, while accounting for only 3% of the number of connections, was responsible for over 40% of the overall bandwidth. They also observed that German users were responsible for over 30% of the requests through their server.

No Legal Review Sought

In his presentation of the work at the PET Symposium yesterday, Kevin Bauer, one of the graduate students who wrote the paper shed some light on the limited amount of legal analysis performed on the project.

Bauer said that the researchers "spoke informally with one lawyer, who told us that that area of the law is ill defined" based on this, the researchers felt that it was "unnecessary to follow up with other lawyers."

The lawyer they spoke to was Professor Paul Ohm, who teaches at the University of Colorado Law School. Ohm has previously collaborated with two of the researchers on an earlier publication, which discussed the legal risks faced by academics engaged network monitoring research. Ohm, a former federal computer crimes prosecutor, has also been the subject of some media attention in recent months, after he publicly stated that ISP-level advertising and traffic-shaping systems may violate US wiretap laws .

In a response to questions by this blogger, Professor Ohm seemed to attempt to distance himself from the researchers, writing by email:

I met with the research team once before they had finished their research, although I don't know how far along they were at that point. At the meeting, I gave them a very brief sketch about federal Wiretap law and they gave me a very brief sketch of their research. They seemed to have put in place a number of controls to try to minimize the risk of liability. I haven't seen the final paper (as far as I can recall).

I'm not their lawyer, and I've never been their lawyer, and I haven't produced any official or unofficial legal advice about their research, but because I spoke with them about this, I don't think it would be appropriate for me to give you any opinions about the research other than this brief statement.

Legal Risks

The Electronic Frontier Foundation, which wrote a legal guide for operators of Tor servers, strongly advises server administrators against snooping on their users. A section in the legal guide makes this clear:

Should I snoop on the plaintext that exits through my Tor relay?

No. You may be technically capable of modifying the Tor source code or installing additional software to monitor or log plaintext that exits your node. However, Tor relay operators in the U.S. can create legal and possibly even criminal liability for themselves under state or federal wiretap laws if they affirmatively monitor, log, or disclose Tor users' communications .... Do not examine the contents of anyone's communications without first talking to a lawyer.

While state laws vary, one immediate concern would be the Wiretap Act, a federal law that broadly prohibits snooping by network operators and others. The core prohibition of the Wiretap Act is found at section 2511(1)(a), which prohibits any person from intentionally intercepting, or attempting to intercept, any wire, oral, or electronic communication." A violation of these rules is is a Class D felony, and can result in fines up to $250,000 and up to 5 years in jail.

It is this same law that groups such as the ACLU and EFF sued AT&T and other telecom companies for violating, when they shared customer communication with the US National Security Agency. AT&T was able to obtain retroactive immunity from the US Congress, but only after spending tens of millions of dollars on lobbyists.

In order to learn more about the legal issues at play, I spoke with Kevin Bankston, the EFF lawyer who wrote the Legal guide for Tor server operators, and who also lead the EFF's lawsuit against AT&T. Bankston told me that:

"I agree that their logging the content exiting their nodes would appear to constitute interceptions of those electronic (not wire) communications under the Wiretap Act, and I don't think they qualify for the narrow provider exceptions [18 USC 2511, 2 (a) I], so I still see the same potential civil and criminal liability that was noted in our FAQ."

No Human Subjects Committee Review

In addition to possible legal issues, the project also raises serious ethical concerns related to the study of users' communications without their consent.

During his presentation, Bauer revealed that the researchers did not seek the approval of their university's Institutional Review Board -- a body that reviews research projects that involve human subjects. He said that, "we were advised that it wasn't necessary," adding that the IRB review process is used "used more in medical and psychology research at our university," and was not generally consulted in computer science projects

Information listed on the website of the University of Colorado's Human Research Committee states that: "All research involving human participants that is conducted by UCB faculty, staff or students must receive some level of review by the Human Research Committee."

Of particular concern to all Institutional Review Boards is any research that involves the study of participants under the age off 18, and other at risk or vulnerable persons. Given that the users of the Tor network have gone out of their way to seek anonymity, and that in some cases, their discovery could lead to arrest or torture, it would seem that these users would almost certainly be considered to be vulnerable. Furthermore, it is quite likely that the snooped communications include at least a few users under the age of 18 -- something that the researchers did not address in their paper.

In a paper published earlier this year, Dr. Simson Garfinkel explored some of the common myths and pitfalls for computer security researchers that study real users and their behavior, and the need to submit their projects to an IRB review.

Dr Garfinkel specifically deals with one of the researcher's claims:

Myth: Because the Common Rule exempts research involving subjects that cannot be identified, IRB approval is not required when using anonymized data

Although this would certainly be convenient, most institutions only allow a determination of exemption to be made by the IRB itself.

A request for clarification on these issues left with the director of the University of Colorado Human Research Committee had not been returned by press time.

Other concerns

In addition to the issues surrounding US legal liability, and ethical concerns over human subject testing -- there is one other problem: International law.

While the researchers are Americans, and conducted their study on a server based in the US, there is certainly an international angle to their study. Users from around the world sent traffic through the researchers' server, and as such more strict Canadian and European intercept and data privacy laws may apply.

Furthermore, one of the strongest privacy protections inherent in the Tor system is the complete lack of logging. That is, if law enforcement agencies approach a Tor server administrator seeking information on a user of the system, the admin can truthfully reply that they have no logs, and thus have nothing that they can be compelled to produce.

Taking questions before their presentation, two of the authors told me that they still have a copy of the data that they collected, and admitted that it was not currently stored on an encrypted disk. They did stress that it was, however, being kept in a "secure" location.

What this means of course, is that law enforcement agencies could easily subpoena this data, thus legally compelling the researchers into handing over the data. This places the users of the Tor network at a significant risk, one that certainly violates the expected social norms of the system.

During the question and answer session after his presentation, Bauer stated that the researchers were still not sure what they were going to do with the data set, and were exploring possibilities for releasing it to researchers in an anonymized and non-personally identifiable way. This statement was met with boos from the audience, which was mainly made up of privacy researchers and activists, a number of whom run their own legitimate Tor servers.

Caveat Emptor

While the US government did not send officials to this annual meeting of privacy researchers, the Canadian government did. A representative for Dr. Ann Cavoukian, the Information and Privacy Commissioner of Ontario was in the audience during the presentation.

When asked for comment on the research project, and any potential impact for Canadian citizens who may have used the snooping Tor server, Cavoukian issued the following statement:

"Whether you run an ISP, a search engine, a Tor server node, or a research project, the principle of Data Minimization should rule. Universal privacy practices require that strong limits be placed on the processing and storage of personal data. In today's online world of constant data availability, privacy requires data minimization at every stage of the information life-cycle: If you don't need the data, don't collect it in the first place; if you don't need it any more, then destroy it securely -- don't keep it any longer than you need to. Full stop."

Wise words indeed.

The major national cable providers are all to sign a troubling yet major censorship deal with a private anti-child porn organization. The deal would give the National Center for Missing and Exploited Children (NCMEC) carte blanche power to issue a takedown of any customer's content hosted on a cable provider's servers.

The group will provide each cable company with a list of Web site addresses that they believe contain child porn. The cable companies will then, per the agreement, scrub the content from their servers.

A press release describing the agreement states that:

The cable operators that have agreed to execute the (memo of understanding) within 30 days include: Comcast Corporation; Cox Communications; Charter Communications; Cablevision Systems Corporation...Time Warner Cable has already signed the MOU.

It is unclear what, if any, notification cable customers will receive before their Web sites are deleted, or what legal rights they will have to appeal the classification of their content as illegal child pornography.

The memo of understanding states that the private group will provide cable companies with a list of kiddie porn URLs, that "in NCMEC's good faith" appears to meet the federal definition of child pornography.

According to Cynthia Brumfield, the industry watcher