FAQ: NSA's data mining explained

A correction was made to this story. Read below for details.

Capitol Hill politicians reacted angrily this week to a new report about how the National Security agency is involved in not merely surveillance of phone calls, but also an extensive data mining program.

"We need to know what our government is doing in its activities that spy upon Americans," said Sen. Patrick Leahy, a Vermont Democrat. Republican Sen. Arlen Specter of Pennsylvania vowed to hold hearings to get to the bottom of how the NSA's data mining works and whether Americans' privacy rights were affected.

To answer some questions about the program and how it likely works, CNET News.com has created the following list of answers to frequently asked questions. Keep reading.

Q: What new information came out this week?
USA Today published an article on Thursday that said AT&T, Verizon and BellSouth turned over records of millions of phone calls to the National Security Agency. These are not international calls--they're apparently records of all calls that those companies' customers made.

Two things are worth noting. First, based on the newspaper's description, contents of phone calls were not divulged. Second, customers' names, street addresses and other personal information were not handed over.

Q: When you say records of phone calls were turned over, what does that mean?
That's a reference to "call detail records," or CDRs, which are database entries that record the parties to the conversation, the duration of the call and so on. This appears to include local phone calls and not just long-distance calls.

CDRs are stored in massive telephone company databases. Cisco Systems' Unified CallManager lets customers use SQL queries to dig up information about each call. Those internal databases have either been opened up to outside queries from the NSA or (more likely) duplicated and handed over to the NSA on a regular basis.

Q: If the NSA has my phone number, can it get my name and address?
Yes. The NSA can cross-check other databases to obtain that information. Many commercial data vendors, such as Yahoo People Search and LexisNexis' People Locator, do just that--and count many federal agencies among their customers.

Q: How about cell phones?
It would be a bit more difficult. There's no central directory for cell phones, for instance. And there's not much information that can be gleaned about owners of disposable cell phones who happened to buy them with cash.

Q: How is this different from what we knew before?
A series of disclosures, starting with The New York Times' report in December, outlined how the NSA conducted surreptitious electronic surveillance of phone calls and e-mail traffic when one party was outside the United States.

The president and other members of his administration have stuck to that claim--saying that domestic phone calls were not part of the dragnet. In January, for instance, Bush assured Americans that "one end of the communication must be outside the United States."

The latest revelation is different. It says the scope of the NSA's efforts is far broader than listening in on a few hundred conversations. Instead, the vast majority of Americans have probably had information on their phone calls turned over. (Another difference is that the contents of the conversations was not divulged, at least as far as we know.)

Q: When Attorney General Alberto Gonzales was testifying a few months ago, he seemed careful to specify that he was talking only about the "Terrorist Surveillance Program." Does that mean he knew about the phone data mining effort and refused to reveal it earlier?
It seems likely, but we don't know. During his appearance before the Senate Judiciary Committee and in a subsequent letter to senators, Gonzales' careful wording seemed to imply that there may be additional domestic surveillance programs beyond the one revealed by The New York Times. (Testifying before senators, Gonzales referred to that program as "the program that the president has confirmed.")

But Gonzales later reassured concerned politicians that the administration is not currently conducting any additional domestic surveillance programs, Rep. Jane Harman, the senior Democrat on the House Intelligence Committee, told The Washington Post in a March interview. Of course, Gonzales could have been parsing his words carefully--and might eventually claim that data mining is not surveillance.

Q: Now that the NSA has this mountain of data, what is the agency doing with it?
The two-word summary: data mining. That's a loose term that generally means directing a computer program to sift through large amounts of data in hopes of extracting previously unknown information.

In theory, useful patterns can emerge and future terrorist plots could be thwarted. In practice, though, The New York Times has reported that FBI sources say many of the tips provided by the NSA led to dead ends.

Q: What other data mining efforts has the NSA been involved with?
Details are classified, of course. But a few hints have become public, and we know that the NSA has funded or been otherwise involved in dozens of programs in the past.

 

Correction: This story mistakenly identified Sen. Bill Nelson as a Florida Republican. He is a Democrat.

[ajbright]

Misinformation

You've said that these records don't include names and addresses or other private information. The reality is that armed with the calling party's phone number retrieving that information is no obstacle to anyone armed with 'net access, Google and perhaps a few other websites that have already admitting selling private financial and phone record related information to law enforcement or other government agencies - despite this being clearly against the law.

Second there is no guarantee that the two incidents you mention are seperated. One could simply be an extension of the other, i.e. if what the EFF claims is true, and has been prevented from disclosing, that millions of domestic phone calls were tapped as well as overseas phonecalls, then a list of who made what calls to whom is extremely useful.

If you know the number of someone you have cause to wiretap, then a list of the phone numbers that have called this number could possibly yield some very useful leads. Once again, call details (having previously worked for a call logging company I have extensive knowledge on how this works) give more than enough information for any government agency to find the very information you say is not included with those records.

Just a quick aside. Senator Spector, the head of the Congressional committee that was supposedly fully informed on both of these issues has some very interesting comments about the whole affair, including the desire to see the public kick up a storm of discontent over both investigations.

To start he say that without doubt the wire tapping issue is in clear violation of FISA, but goes on to say that Bush may have had reasonable cause for carrying out both initiatives.

The interesting item was the he said although it may be true that Bush had reasonable cause and it may be true that Bush had authority to sanction both initiatives, because he doesn't know any of the details of either issue, he can't be sure that what Bush has done is legal or otherwise.

On the wire tapping issue it's also interesting to note that a Dept. of Justice enquiry into that particular affair was shut down after they were denied access to the very information they needed to prove whether or not Bush was telling us the truth when he stated the wire tapping was extremely limited, targetting only those individuals that were known to have links to terrorism.

Finally in answer to those that suggest all Americans are terrorists (because apparently the call detail records of tens of millions of Americans were required to fight terrorism) and that there is no need to investigate either matter I would echo another of their very own arguments as to why I should not object to Bush and his cronies from listening in on my personal calls to family and friends overseas or otherwise.

"If you have nothing to hide, you have nothing to fear" so why are you using all resources at your disposal to prevent any sort of investigation from being completed?

Those in charge of these investigations are high ranking, high security clearance members of the Justice Department as well as the oversight committee members of the Intelligence Community. The argument that an investigation by these parties would compromise national security is completely false as both are fully entitled to know everything that the intelligence agencies do with the money the tax payers give them.

[Razzl]

Public reaction may not hold

While it was unexpected and disconcerting to see a majority of the public polled saying they didn't have a problem with this program, nothing has been indicated yet about what kinds of questions people were really asked and what details people really knew. Certainly Congressmen and those of us who object to this and see the danger of this whole package of behavior should continue to speak out strongly against it whatever majority opinion may say; after all, the Iraq war was clearly wrong and illegal and by minority voices speaking out the public has been educated to the problem and changed its opinion on that issue.

But I suspect the public will turn around very quickly on this one as the details sink in. It won't take a year for people to realize how outrageous, excessive, dangerous, and as always, unproductive this technique is, and how corrosive to their rights in the long run.

[longofest]

Big Distinction: NSA is NOT eavesdropping (in this program)

The biggest misconception that people are getting about this program from just reading the headlines and not reading any further into the articles is that the NSA is wiretapping and storing the contents of every phone call made in the US. This is plain false. The article here did an okay job of clarifying this, but really, that point must be made crystal clear.

The program that has been listed is aimed solely at identifying calling patterns of terrorists, so that previously unknown sleeper cells can be identified. Think about it. The NSA/CIA/FBI knows about certain sleeper cells and those individuals. If the NSA can figure out what the communication patters of a subversive terrorist are like, then they could potentially identify other similar patterns and catch previously unknown sleeper cells. Of course, in order to fully realize this, they would have to then eavesdrop on the contents of a suspect's conversation, which they would need to get a warrant to do for it to be legal.

I have had beef with Bush circumventing the warrant system for the international calls, but since these calls aren't actually tapped but just logged, I think it's actually a good idea. As long as warrants are gotten to actually tap the phones after suspects are identified and the FISA court is involved, this program could be a great tool to keep 911 from happening again!

NSA Data Mining Explained

I am grateful that my government is doing all it can to keep my loved ones safe....I am not giving up a right,,,I am giving my permission, why shouldnt we all stick together as Americans and give our government a break....dont get in the way of them trying to keep us safe, dont tie up resources...if you have nothing to hide,,,,there is no reason to cry

NSA Data Mining Explained

I am grateful that my government is doing all it can to keep my loved ones safe....I am not giving up a right,,,I am giving my permission, why shouldnt we all stick together as Americans and give our government a break....dont get in the way of them trying to keep us safe, dont tie up resources...if you have nothing to hide,,,,there is no reason to cry

No Names and No Wiretapping, Don't Breath A Sigh of Relief

Whether or not they provided names and addresses is rather
immaterial. Getting a name an address from a phone number is
trivial for the average citizen, never mind the NSA.

Similarly, the difference (to you as a citizen concerned about
your rights, as well as to a potential terrorist) between "call
records" and "wire tapping" is also pretty immaterial. The NSA's
speciality is traffic analysis. Military and other traffic that they
study has been encrypted to years--that doesn't put them out of
business. On the contrary, knowing who is talking to whom
gives them 90% of what they want to know.

Let's get specific.

Would you want your employer to know that you make regular
calls to an AIDs Test Center?

You work for a small company whose owner is a rabid
Republican. Want him to know that you regularly call a
Democratic donation center?

You work part time in the phone sex industry, some of your
clients take call backs, including some politicians. Want those
records available?

All of those things make you subject to political pressure.
Having someone knows those things puts your career in
jeopardy. The government has used information like that in the
past to pressure everyone from private citizens to politicians.
How much do you want to gamble that there isn't *someone*
who has access to that information who can't be bribed to turn
over some "harmless" information to someone in a position to
use it?

When it comes down it to it, you'd almost *rather* that they
handed over wiretaps. Then at least you'd have the ability to
prove that you weren't doing what they implied from the calls.
Without that information, you don't need to *be* guilty of
anything, you just need to *look* guilty.

[wtortorici]

NSA has no legal right to domestic data

NSA was setup to spy or listen to non-domestic communications. No matter that no actual conversations or names are included they have no business delving into domestic transactions.

It doesn't take much more effort for any agency to piece together names, numbers and data of all citizens. Now put that information into the hands of uncontrolled agency like the NSA and even if you never did anything wrong in your life, how simple would it be to add to your file that your a pedafile, a bank robber or a murderer.

You think it's far fletched, how many inoccent people have been convicted of those crimes by over zealous DAs and detectives. Now put all that data in their hands and you'll spend the rest of your life in prison or executed.

Have a nice day.

[jfmezei]

Old news.

After 9-11, the USA government outline its goal of "Total information awareness" and many parts were included in the patriot act that was blindly signed into law by your elected officials.

Ever politician who voted for the patriot act and other police-state laws has no moral right to criticise the laws he voted for.

They should be held accountable for voting into law a huge bills that they haven't taken the time to really read and debate.

The USA media is also at fault for not having gone through the act and found all the clauses that turn the USA into a police state and challenge every polician to explain why he/she voted for such clauses.

Not only was the act enacted, but americans re-elected the government that did it.

[scdecade]

Congress

Members of Congress aren't mad that the government is confiscating records from private companies without reason for suspicion. Haha, only a complete ignoramus would think that. Congress is mad that there weren't TOLD about it. Big distinction there. Congress doesn't care about due process of the law. Congress cares about power and the ability to capricously use information for whatever purposes they deem politically desirable to themselves.

[rwhitr]

Data Mining

I am pleased to see the NSA using its intelligence to glean information from the vast database that might help to thwart a domestic terrorist incident. If I were a QWest customer or stockholder I would be annoyed that my company was not participating. I am sick and tired of hearing people complain that their government is not doing enough to protect them, and then complaining louder when they make use of analysis of raw data to aid in stopping the terrorists. Be thankful they are working in your behalf!

[rwhitr]

Data Mining

I am pleased to see the NSA using its intelligence to glean information from the vast database that might help to thwart a domestic terrorist incident. If I were a QWest customer or stockholder I would be annoyed that my company was not participating. I am sick and tired of hearing people complain that their government is not doing enough to protect them, and then complaining louder when they make use of analysis of raw data to aid in stopping the terrorists. Be thankful they are working in your behalf!

[mozartsbum]

Thanks for the article

Since the USA is now a totalitarian state now, the federal government and many state governments having been taken over by the corporations, neocons and the tyrants running the White House, the Congress, the Supreme Court and both political parties, we need more articles like yours. as Fredrick Douglass said "Knowledge makes a man unfit to be a slave.". However: "The truth is that men are tired of liberty."-Mussolini. "Big money and big business, corporations and commerce, are again the undisputed overlords of politics and government. The White House, the Congress and, increasingly, the judiciary, reflect their
interests. We appear to have a government run by remote control from the U.S. Chamber of Commerce, the National Association of Manufacturers and the American Petroleum Institute. To hell with everyone else." -Bill Moyers - PBS Commentator
""Information is the currency of Democracy." "In matters of style, swim with the current, in matters of principle, stand like a rock" "If all the people knew all the facts, they would never make a mistake." "It is better for one hundred guilty men to go free than one innocent man to go to jail" "It is wrong to take a man's money and use it to promote ideas he does not agree with" "It's better to debate an issue without settling it, than to settle an issue without debate." "The end of democracy, and the defeat of the American Revolution will occur when government falls into the hands of the lending institutions and moneyed incorporations." : Thomas Jefferson
As Pericles said in 430 B.C.: "Just because you do not take an interest in politics doesn't mean politics won't take an interest in you."

cheers all!
Stephen Bennett
Paineitte and True Patriot

[Stalin Hornsby]

Very very old news. too date only the CIA and Barbera Bush can know.

Linden Johnson knew. Jack Kenedey and Robert Kenedey both were working on it. But, fashion was clothing and Jesus Christ was a corse of action. Now even Very very has Osmodius' eyes all over the alpha beta base. Don't mess with Texas and Area 51.

[TheZooker]

So-What?

There are many more areas of personal information which are available to Government as well as anyone else wanting to know. Why do we allow ourselves to be so hyped as to feel we somehow have been "spied" upon by the release of this information. The IRS has more on us that the phone data bank will ever be able to discover! It is all a political hype and is nonsense!

[ajbright]

It's all our own fault..

Should we really be surprised by any of this? For years we've been telling G.W. we needed more intelligence in the White House.. is it any surprise he just got things a little mixed up?

Also I find the rumours he holds his current job due to an argument his mom had with his dad to be extremely unlikey.. despite several people claiming to hear as she stormed out "any idiot can become president"..!!