May 12, 2006 1:20 PM PDT

FAQ: NSA's data mining explained

A correction was made to this story. Read below for details.

Capitol Hill politicians reacted angrily this week to a new report about how the National Security agency is involved in not merely surveillance of phone calls, but also an extensive data mining program.

"We need to know what our government is doing in its activities that spy upon Americans," said Sen. Patrick Leahy, a Vermont Democrat. Republican Sen. Arlen Specter of Pennsylvania vowed to hold hearings to get to the bottom of how the NSA's data mining works and whether Americans' privacy rights were affected.

To answer some questions about the program and how it likely works, CNET News.com has created the following list of answers to frequently asked questions. Keep reading.

Q: What new information came out this week?
USA Today published an article on Thursday that said AT&T, Verizon and BellSouth turned over records of millions of phone calls to the National Security Agency. These are not international calls--they're apparently records of all calls that those companies' customers made.

Two things are worth noting. First, based on the newspaper's description, contents of phone calls were not divulged. Second, customers' names, street addresses and other personal information were not handed over.

Q: When you say records of phone calls were turned over, what does that mean?
That's a reference to "call detail records," or CDRs, which are database entries that record the parties to the conversation, the duration of the call and so on. This appears to include local phone calls and not just long-distance calls.

CDRs are stored in massive telephone company databases. Cisco Systems' Unified CallManager lets customers use SQL queries to dig up information about each call. Those internal databases have either been opened up to outside queries from the NSA or (more likely) duplicated and handed over to the NSA on a regular basis.

Q: If the NSA has my phone number, can it get my name and address?
Yes. The NSA can cross-check other databases to obtain that information. Many commercial data vendors, such as Yahoo People Search and LexisNexis' People Locator, do just that--and count many federal agencies among their customers.

Q: How about cell phones?
It would be a bit more difficult. There's no central directory for cell phones, for instance. And there's not much information that can be gleaned about owners of disposable cell phones who happened to buy them with cash.

Q: How is this different from what we knew before?
A series of disclosures, starting with The New York Times' report in December, outlined how the NSA conducted surreptitious electronic surveillance of phone calls and e-mail traffic when one party was outside the United States.

The president and other members of his administration have stuck to that claim--saying that domestic phone calls were not part of the dragnet. In January, for instance, Bush assured Americans that "one end of the communication must be outside the United States."

The latest revelation is different. It says the scope of the NSA's efforts is far broader than listening in on a few hundred conversations. Instead, the vast majority of Americans have probably had information on their phone calls turned over. (Another difference is that the contents of the conversations was not divulged, at least as far as we know.)

Q: When Attorney General Alberto Gonzales was testifying a few months ago, he seemed careful to specify that he was talking only about the "Terrorist Surveillance Program." Does that mean he knew about the phone data mining effort and refused to reveal it earlier?
It seems likely, but we don't know. During his appearance before the Senate Judiciary Committee and in a subsequent letter to senators, Gonzales' careful wording seemed to imply that there may be additional domestic surveillance programs beyond the one revealed by The New York Times. (Testifying before senators, Gonzales referred to that program as "the program that the president has confirmed.")

But Gonzales later reassured concerned politicians that the administration is not currently conducting any additional domestic surveillance programs, Rep. Jane Harman, the senior Democrat on the House Intelligence Committee, told The Washington Post in a March interview. Of course, Gonzales could have been parsing his words carefully--and might eventually claim that data mining is not surveillance.

Q: Now that the NSA has this mountain of data, what is the agency doing with it?
The two-word summary: data mining. That's a loose term that generally means directing a computer program to sift through large amounts of data in hopes of extracting previously unknown information.

In theory, useful patterns can emerge and future terrorist plots could be thwarted. In practice, though, The New York Times has reported that FBI sources say many of the tips provided by the NSA led to dead ends.

Q: What other data mining efforts has the NSA been involved with?
Details are classified, of course. But a few hints have become public, and we know that the NSA has funded or been otherwise involved in dozens of programs in the past.

 

Correction: This story mistakenly identified Sen. Bill Nelson as a Florida Republican. He is a Democrat.

CONTINUED: Is this type of data mining legal?…
Page 1 | 2 | 3

See more CNET content tagged:
NSA, data mining, telephone call, BellSouth Corp., FAQ

37 comments

Join the conversation!
Add your comment
Misinformation
You've said that these records don't include names and addresses or other private information. The reality is that armed with the calling party's phone number retrieving that information is no obstacle to anyone armed with 'net access, Google and perhaps a few other websites that have already admitting selling private financial and phone record related information to law enforcement or other government agencies - despite this being clearly against the law.

Second there is no guarantee that the two incidents you mention are seperated. One could simply be an extension of the other, i.e. if what the EFF claims is true, and has been prevented from disclosing, that millions of domestic phone calls were tapped as well as overseas phonecalls, then a list of who made what calls to whom is extremely useful.

If you know the number of someone you have cause to wiretap, then a list of the phone numbers that have called this number could possibly yield some very useful leads. Once again, call details (having previously worked for a call logging company I have extensive knowledge on how this works) give more than enough information for any government agency to find the very information you say is not included with those records.

Just a quick aside. Senator Spector, the head of the Congressional committee that was supposedly fully informed on both of these issues has some very interesting comments about the whole affair, including the desire to see the public kick up a storm of discontent over both investigations.

To start he say that without doubt the wire tapping issue is in clear violation of FISA, but goes on to say that Bush may have had reasonable cause for carrying out both initiatives.

The interesting item was the he said although it may be true that Bush had reasonable cause and it may be true that Bush had authority to sanction both initiatives, because he doesn't know any of the details of either issue, he can't be sure that what Bush has done is legal or otherwise.

On the wire tapping issue it's also interesting to note that a Dept. of Justice enquiry into that particular affair was shut down after they were denied access to the very information they needed to prove whether or not Bush was telling us the truth when he stated the wire tapping was extremely limited, targetting only those individuals that were known to have links to terrorism.

Finally in answer to those that suggest all Americans are terrorists (because apparently the call detail records of tens of millions of Americans were required to fight terrorism) and that there is no need to investigate either matter I would echo another of their very own arguments as to why I should not object to Bush and his cronies from listening in on my personal calls to family and friends overseas or otherwise.

"If you have nothing to hide, you have nothing to fear" so why are you using all resources at your disposal to prevent any sort of investigation from being completed?

Those in charge of these investigations are high ranking, high security clearance members of the Justice Department as well as the oversight committee members of the Intelligence Community. The argument that an investigation by these parties would compromise national security is completely false as both are fully entitled to know everything that the intelligence agencies do with the money the tax payers give them.
Posted by ajbright (447 comments )
Reply Link Flag
Public reaction may not hold
While it was unexpected and disconcerting to see a majority of the public polled saying they didn't have a problem with this program, nothing has been indicated yet about what kinds of questions people were really asked and what details people really knew. Certainly Congressmen and those of us who object to this and see the danger of this whole package of behavior should continue to speak out strongly against it whatever majority opinion may say; after all, the Iraq war was clearly wrong and illegal and by minority voices speaking out the public has been educated to the problem and changed its opinion on that issue.

But I suspect the public will turn around very quickly on this one as the details sink in. It won't take a year for people to realize how outrageous, excessive, dangerous, and as always, unproductive this technique is, and how corrosive to their rights in the long run.
Posted by Razzl (1318 comments )
Reply Link Flag
Mislead
The problem is that all the major networks are misleading the public by stating that names and addresses were not included in the records handed over.

Any reporter worth his or her salt would know that names and addresses are extremely easy to obtain if you have a list of phone numbers.

Unfortunately as usual our so-called liberal media is ignoring a blatently obvious fact, because they are too spineless to call something for what it is.

They were too spineless over the lies that led to the war in Iraq, too spineless to ask why we took troops away from hunting down Al Qaeda and Osama Bin Laden, too spineless to question why the NSA is wire tapping and why the Justice Department has been deliberately prevented from finding out details of that so-called national security issue.

So it shouldn't be much of a surprise that a media more interested in which direction Michael Jackson's car is headed than actual news, hasn't got the balls to state that the Administration is clearly misleading the public when it says that names and addresses are included in the information it gathers.

Until our media has the balls to tell our politicians that they know they're lying, we haven't got a hope of the general public raising any sort of voting rebellion against congress.
Posted by ajbright (447 comments )
Link Flag
Big Distinction: NSA is NOT eavesdropping (in this program)
The biggest misconception that people are getting about this program from just reading the headlines and not reading any further into the articles is that the NSA is wiretapping and storing the contents of every phone call made in the US. This is plain false. The article here did an okay job of clarifying this, but really, that point must be made crystal clear.

The program that has been listed is aimed solely at identifying calling patterns of terrorists, so that previously unknown sleeper cells can be identified. Think about it. The NSA/CIA/FBI knows about certain sleeper cells and those individuals. If the NSA can figure out what the communication patters of a subversive terrorist are like, then they could potentially identify other similar patterns and catch previously unknown sleeper cells. Of course, in order to fully realize this, they would have to then eavesdrop on the contents of a suspect's conversation, which they would need to get a warrant to do for it to be legal.

I have had beef with Bush circumventing the warrant system for the international calls, but since these calls aren't actually tapped but just logged, I think it's actually a good idea. As long as warrants are gotten to actually tap the phones after suspects are identified and the FISA court is involved, this program could be a great tool to keep 911 from happening again!
Posted by longofest (13 comments )
Reply Link Flag
Maybe, but
I think most people easily make the jump that if Bush is collecting all this data, Bush is listening in as well. The thing is there is no "probable cause" for tapping and keeping records on every American, or even a significant portion.

It comes down to this, if they need the records of everyone, they have no clue who the bad guys are. And if they don't know who the bad guys are, we're screwed.
Posted by R. U. Sirius (745 comments )
Link Flag
Bush doesn't care about the FISA court
That should be pretty obvious by now. Check the earlier stories.
NSA approached Qwest about turning over their customer records.
Qwest referred them to their attorneys. Those attorneys were
concerned that it could be illegal and could create a liability
for Qwest, so they asked NSA to simply use the FISA court as the
law already allows them to do.
Posted by Jackson Cracker (272 comments )
Link Flag
Yes and no.
I think people are clear that the NSA has
received copies of everyone's phone records from
the companies. I don't know of other companies,
but Verizon customers would note that their
complicity is a violation of your service
contract with them (periodically they send out a
privacy statement with your bill, go through
your records for it).

The questions are -- was it legal? What are they
doing with the information? And, why does the
protocol they are following implicate everyone
in their surveillance rather than just suspects?

Was it legal? Well, not by the letter of the
law, even those condoning it agree on that
point. Does the president have the authority to
authorize the violation of the law in the name
of national security? Well, that's the debate
we're having now. Bush has publicly "set
aside" (written a statement stating he asserts
presidential privilege to ignore and authorize
other to ignore) about 400 laws that he feels
interferes with the activities of his
administration.

What are they doing with the information? Who
knows. Ostensibly, it's to track "terrorists",
but the term is used pretty broadly. For
example, a Quaker church group in Florida
appeared on a leaked DoD memo citing it as a
terrorist group for protesting military
recruitment at local high-schools. This would be
in line with the McCarthy and J.E.Hoover
interpretations, and presumably that of Gonzalez
as well. The information itself doesn't make
sense the the context of locating terrorists
because the records they got were for a broad
swath of domestic numbers which they already
know (through access to other records) belong to
average US citizens (and because the guys they
tell us they're after still aren't big on using
POTS phones). Whatever they are doing, the
information is being used to profile US citizens
and not "terrorists" in the sense of Al Qaeda.
That's not to say that there might be an effort
to identify a "lunatic fringe" or some such
thing that might get some ideas... Who knows.
Perhaps it's all just a ruse to either make
would-be evil-doers lives more complex because
hiding their phone activity is just one other
thing they need to think of (doubtful, they most
assuredly know that by now). Or, perhaps it's
just a test to see how much people really value
their privacy. It's not as if it'll ever be
restored, easing people into being comfortable
with being constantly monitored and their
records and possessions searched takes time.
Posted by Gleeplewinky (289 comments )
Link Flag
NSA Data Mining Explained
I am grateful that my government is doing all it can to keep my loved ones safe....I am not giving up a right,,,I am giving my permission, why shouldnt we all stick together as Americans and give our government a break....dont get in the way of them trying to keep us safe, dont tie up resources...if you have nothing to hide,,,,there is no reason to cry
Posted by (5 comments )
Reply Link Flag
You have to be kidding...
By your reasoning it's ok to kill american men and women in Iraq so they won't be killed on United States soil.

Give up your rights so your children will be able to grow up to die in a fight against terrorist. There is no compromise on freedom, either your willing to stand up for all of them or none.

Have a nice day. ;)
Posted by wtortorici (102 comments )
Link Flag
Well...If you really have nothing to hide...
Be so kind as to post your real name and home address. I would like to stop by an wander through your house to see what kind of "stuff" you have. Don't worry, I won't take anything. I just want to look. Oh...And please be sure to have your latest bank statements and tax records. I'll want to look at those too. I'm sure you also won't mind if I bring a couple thousand of my furthest strangers with me as guests. After all, you have nothing to hide...right?
;-)
Posted by (63 comments )
Link Flag
NSA Data Mining Explained
I am grateful that my government is doing all it can to keep my loved ones safe....I am not giving up a right,,,I am giving my permission, why shouldnt we all stick together as Americans and give our government a break....dont get in the way of them trying to keep us safe, dont tie up resources...if you have nothing to hide,,,,there is no reason to cry
Posted by (5 comments )
Reply Link Flag
No Names and No Wiretapping, Don't Breath A Sigh of Relief
Whether or not they provided names and addresses is rather
immaterial. Getting a name an address from a phone number is
trivial for the average citizen, never mind the NSA.

Similarly, the difference (to you as a citizen concerned about
your rights, as well as to a potential terrorist) between "call
records" and "wire tapping" is also pretty immaterial. The NSA's
speciality is traffic analysis. Military and other traffic that they
study has been encrypted to years--that doesn't put them out of
business. On the contrary, knowing who is talking to whom
gives them 90% of what they want to know.

Let's get specific.

Would you want your employer to know that you make regular
calls to an AIDs Test Center?

You work for a small company whose owner is a rabid
Republican. Want him to know that you regularly call a
Democratic donation center?

You work part time in the phone sex industry, some of your
clients take call backs, including some politicians. Want those
records available?

All of those things make you subject to political pressure.
Having someone knows those things puts your career in
jeopardy. The government has used information like that in the
past to pressure everyone from private citizens to politicians.
How much do you want to gamble that there isn't *someone*
who has access to that information who can't be bribed to turn
over some "harmless" information to someone in a position to
use it?

When it comes down it to it, you'd almost *rather* that they
handed over wiretaps. Then at least you'd have the ability to
prove that you weren't doing what they implied from the calls.
Without that information, you don't need to *be* guilty of
anything, you just need to *look* guilty.
Posted by (10 comments )
Reply Link Flag
Oh humbug, a typo in the headline
and no "edit" option. Sorry about that.
Posted by (10 comments )
Link Flag
As Confused as they come
Your argument simply makes no sense.

First of all, the NSA isn't distributing call data to your employer. Of course, they have no reason to do so, because in the examples you provided, your employer has the RIGHT to record your calls and conversations and fire you if they so choose. Government need not be involved.

Paranoia, is the problem. There is not one known abuse of this system. Yes, numbers can be translated into names... and that's GOOD. How else do you figure out who the terrorist on the phone is? Sheesh. It isn't like the government doesn't have your name, phone number, address, ssn, date of birth... on and on and on... already. If they wanted to troll to bust you for something trivial, they already could... but its ILLEGAL. There is no expectation of privacy on the call records that are being collected. Countless employees at your phone company have access to those records, and they are printed on every bill you receive.

You spout paranoia and frightened speculation. But no evidence of any wrongdding. YOu don't have to like what your governmetn does, but suggesting that they are evil and illegal simply because you don't like it, is irresponsible and immature.
Posted by David Arbogast (1709 comments )
Link Flag
NSA has no legal right to domestic data
NSA was setup to spy or listen to non-domestic communications. No matter that no actual conversations or names are included they have no business delving into domestic transactions.

It doesn't take much more effort for any agency to piece together names, numbers and data of all citizens. Now put that information into the hands of uncontrolled agency like the NSA and even if you never did anything wrong in your life, how simple would it be to add to your file that your a pedafile, a bank robber or a murderer.

You think it's far fletched, how many inoccent people have been convicted of those crimes by over zealous DAs and detectives. Now put all that data in their hands and you'll spend the rest of your life in prison or executed.

Have a nice day. ;)
Posted by wtortorici (102 comments )
Reply Link Flag
Paranoia.
First, the NSA's sole responsibility is not listening to foreign communications. Educate yourself on this highly technical and important government body before you try to explain to other people why it exists.

Secondly, there is no expectation of privacy regarding call logs. Your number, the number you called, and the duration of the call. Its on every bill you receive and is not protected data.

Yes, the government can look up your name and address with the phone numbers they collect. Guess what? The government has your name and address already. Sheesh... Social Security? Internal Revenue Service? You think you can hide from your government? HA!

You haven't mentioned one shred of evidence that points to an abuse or even a single problem with this particular data collection. You spout speculations and paranoia...

I will have a nice day... becasue I know that my government is working to keep me safe. Yes... they could do better, but attacking the NSA at this point is regressive.

In fact, Attacking the NSA program is a blind attack against the administration with no regard for national security. It doesn't surprise me that some people do it... heck.. some people will even vote for a president based on an "anybody but" approach. Talk about ignorant!
Posted by David Arbogast (1709 comments )
Link Flag
Old news.
After 9-11, the USA government outline its goal of "Total information awareness" and many parts were included in the patriot act that was blindly signed into law by your elected officials.

Ever politician who voted for the patriot act and other police-state laws has no moral right to criticise the laws he voted for.

They should be held accountable for voting into law a huge bills that they haven't taken the time to really read and debate.

The USA media is also at fault for not having gone through the act and found all the clauses that turn the USA into a police state and challenge every polician to explain why he/she voted for such clauses.

Not only was the act enacted, but americans re-elected the government that did it.
Posted by jfmezei (24 comments )
Reply Link Flag
Your right about our congress delegating their
responsabilities to the office staff who could give a **** if you and I live in a police state. Their only concern is to get re-elected and get their retirement benifts. So send our kids, sons and daughters to foriegn lands to get killed and maimed. More of our children have been killed and injured in Iraq than from the 9/11 attack. But atleast there were no attacks and deaths on American soil. That's little comfort to the families of those who have been sacrificed.

It pisses me off, but Washington is so corrupt, it doesn't even know it's corrupt.

It took 4-1/2 years for the majority to realize that Bush/Cheney are liars and will do and say anything to continue. They have no shame or compassion for us working stiffs, only for their money cronnies.
Posted by wtortorici (102 comments )
Link Flag
Congress
Members of Congress aren't mad that the government is confiscating records from private companies without reason for suspicion. Haha, only a complete ignoramus would think that. Congress is mad that there weren't TOLD about it. Big distinction there. Congress doesn't care about due process of the law. Congress cares about power and the ability to capricously use information for whatever purposes they deem politically desirable to themselves.
Posted by scdecade (329 comments )
Reply Link Flag
Data Mining
I am pleased to see the NSA using its intelligence to glean information from the vast database that might help to thwart a domestic terrorist incident. If I were a QWest customer or stockholder I would be annoyed that my company was not participating. I am sick and tired of hearing people complain that their government is not doing enough to protect them, and then complaining louder when they make use of analysis of raw data to aid in stopping the terrorists. Be thankful they are working in your behalf!
Posted by rwhitr (4 comments )
Reply Link Flag
sick and tired of hearing people complain
<a class="jive-link-external" href="http://www.analogstereo.com/subaru_forester_owners_manual.htm" target="_newWindow">http://www.analogstereo.com/subaru_forester_owners_manual.htm</a>
Posted by Ipod Apple (152 comments )
Link Flag
Data Mining
I am pleased to see the NSA using its intelligence to glean information from the vast database that might help to thwart a domestic terrorist incident. If I were a QWest customer or stockholder I would be annoyed that my company was not participating. I am sick and tired of hearing people complain that their government is not doing enough to protect them, and then complaining louder when they make use of analysis of raw data to aid in stopping the terrorists. Be thankful they are working in your behalf!
Posted by rwhitr (4 comments )
Reply Link Flag
Thanks for the article
Since the USA is now a totalitarian state now, the federal government and many state governments having been taken over by the corporations, neocons and the tyrants running the White House, the Congress, the Supreme Court and both political parties, we need more articles like yours. as Fredrick Douglass said "Knowledge makes a man unfit to be a slave.". However: "The truth is that men are tired of liberty."-Mussolini. "Big money and big business, corporations and commerce, are again the undisputed overlords of politics and government. The White House, the Congress and, increasingly, the judiciary, reflect their
interests. We appear to have a government run by remote control from the U.S. Chamber of Commerce, the National Association of Manufacturers and the American Petroleum Institute. To hell with everyone else." -Bill Moyers - PBS Commentator
""Information is the currency of Democracy." "In matters of style, swim with the current, in matters of principle, stand like a rock" "If all the people knew all the facts, they would never make a mistake." "It is better for one hundred guilty men to go free than one innocent man to go to jail" "It is wrong to take a man's money and use it to promote ideas he does not agree with" "It's better to debate an issue without settling it, than to settle an issue without debate." "The end of democracy, and the defeat of the American Revolution will occur when government falls into the hands of the lending institutions and moneyed incorporations." : Thomas Jefferson
As Pericles said in 430 B.C.: "Just because you do not take an interest in politics doesn't mean politics won't take an interest in you."

cheers all!
Stephen Bennett
Paineitte and True Patriot
Posted by mozartsbum (6 comments )
Reply Link Flag
Very very old news. too date only the CIA and Barbera Bush can know.
Linden Johnson knew. Jack Kenedey and Robert Kenedey both were working on it. But, fashion was clothing and Jesus Christ was a corse of action. Now even Very very has Osmodius' eyes all over the alpha beta base. Don't mess with Texas and Area 51.
Posted by Stalin Hornsby (60 comments )
Reply Link Flag
So-What?
There are many more areas of personal information which are available to Government as well as anyone else wanting to know. Why do we allow ourselves to be so hyped as to feel we somehow have been "spied" upon by the release of this information. The IRS has more on us that the phone data bank will ever be able to discover! It is all a political hype and is nonsense!
Posted by TheZooker (2 comments )
Reply Link Flag
It's all our own fault..
Should we really be surprised by any of this? For years we've been telling G.W. we needed more intelligence in the White House.. is it any surprise he just got things a little mixed up?

Also I find the rumours he holds his current job due to an argument his mom had with his dad to be extremely unlikey.. despite several people claiming to hear as she stormed out "any idiot can become president"..!!
Posted by ajbright (447 comments )
Reply Link Flag
 

Join the conversation

Add your comment

The posting of advertisements, profanity, or personal attacks is prohibited. Click here to review our Terms of Use.

What's Hot

Discussions

Shared

RSS Feeds

Add headlines from CNET News to your homepage or feedreader.