The NSA Reveals That It Does 20 Million Database Queries Per Month
from the that's-a-lot dept
As we noted earlier today, the NSA’s two key “defenses” of the thousands of abuses and violations of the law that recently came out thanks to a leaked document are that there wasn’t “intent” to abuse the system (we had no idea that made illegal things legal…) and, second, that it was such a small percentage of the activity that it’s really no big deal. Glenn Greenwald quickly noted that the NSA is actually saying “we collect billion of emails and calls every day, so what’s a few thousand privacy violations?” hoping that everyone focuses on the second half of the sentence. But the key point is actually the first half of that sentence. In fact, as we noted in that last post, the NSA’s top compliance guy actually revealed a startling fact in his attempt to push the meaningless “ratio” of violations to queries:
The official, John DeLong, the N.S.A. director of compliance, said that the number of mistakes by the agency was extremely low compared with its overall activities. The report showed about 100 errors by analysts in making queries of databases of already-collected communications data; by comparison, he said, the agency performs about 20 million such queries each month.
Again, the ratio is a meaningless number. You’re not declared innocent of murder because you didn’t happen to murder someone every other day of your life. But, perhaps more important in this is the revelation of the 20 million queries every single month. Or, approximately 600,000 queries every day. How about 25,000 queries every hour? Or 417 queries every minute? Seven queries every single second. Holy crap, that’s a lot of queries.
Remember, too, that the NSA has insisted that it doesn’t datamine its data collection, which is clearly hogwash. That many queries means they’re trolling through that database all the time. Remember how the NSA was trying to play down how often it did queries by saying that only 300 phone numbers had been used to “initiate” a query? Yeah, well, once again, it would appear that the NSA was not being fully forthcoming about these sorts of things. Shocking, I know, but I’d imagine they’d claim it was the “least untruthful” answer they could come up with after having a good week or so to answer the question.
Filed Under: database queries, datamining, nsa, nsa surveillance, privacy
Comments on “The NSA Reveals That It Does 20 Million Database Queries Per Month”
Well, if you consider Snowden came from the NSA and he’s letting all this in the wild then the NSA is indirectly telling the truth. O.o
And some of the basic infra-structure of the web is in American soil. Sure this mass surveillance on Americans is worrisome for the Americans themselves but shouldn’t the rest of the world be worried too?
Well, to be honest that’s not a lot considering the US has over 300 million terrorists inside it’s borders.
Re: Re:
That’s about .002% of the population?
Er.. I need to stop posting stupid shit like this before the NSA decides to recruit me as an numbers analyst.
SELECT * FROM Public_Data_All
–WHERE IsTerrorist = 1
Re: Re:
Where is the mapreduce of that one?
Re: Re:
SELECT * FROM Public_Data_All
WHERE last_name like ‘Snowde%’
FTFY
Re: Re:
SELECT * FROM Public_Data_All
–WHERE IsTerrorist = 1
Commenting out the where clause explains why they think every American is a terrorist.
Re: Re:
We should replace some commonly used word for terrorist for the lulz. Ie: reader
Techdirt terrorists often discuss about tech stuff. According to these terrorists technology can make carrying their daily terrorism (reading) tasks easier. Filter gone haywire!
Re: Re:
SELECT * FROM PublicData
WHERE Exists = 1
How many of those use "direct access" to Google's servers?
Which is worse: NSA or the collaborating corporations?
And which spying corporation is the worst? Google, Facebook, Microsoft, Yahoo, Apple…
Re: How many of those use "direct access" to Google's servers?
“Which is worse: NSA or the collaborating corporations?”
Two sides of the same coin. The NSA uses the private corporations to harvest data on the population.
Corporate spying alone is way down the list because people have the choice as to whether or not they want to use that company’s services.
I don’t know why I bothered because you will inevitably ignore what I said and continue with your anti corporation rants.
Re: Re: Corporate money out of politics
Being anti-corporate is not exactly irrational. Partially misdirected blame but even if a person is completely “anti-corporation”, is it really that bad of an ideology ?
OBLIGATION of business to make profit disregarding any societal cost.
Without regulation and laws then growing people to be slaves would be legitimate pro-corporation ideology.
It’s the laws and regulations that need to be the focus.
By law, google are not allowed to grow slaves.
By law, google are required to hand over all access to NSA.
Who is funding the American Laws ?
Corporations.
So being anti-corporation is actually legitimate.
Being anti corporate funding of politics is smarter.
Until the money is out of politics, there will be a valid reason for people to be anti-corporatist.
Re: How many of those use "direct access" to Google's servers?
You forgot the companies of the industrial military complex.
Corporations ——–> $$$$$$$$ ———> Government Officials
Government Officials ——–> Beneficial Laws ———> Corporations
Google, Facebook, Microsoft, Yahoo, Apple etc… are pretty much victims and are just complying with these new “Laws”.
It’s the Corporations that are developing/supplying Tech and services for the NSA that are the ones making the laws via bribery.
Re: Re: How many of those use "direct access" to Google's servers?
Corporations ——–> $$ ———> Government Officials
Government Officials ——–> Beneficial Laws ———> Corporations——> Hire Government Officials upon leaving office or positions for $$$$$$$$$$$$$$
Fixed
Re: How many of those use "direct access" to Google's servers?
“Which is worse: NSA or the collaborating corporations?”
Fucking idiot. The fact that you even have to ask that shows your bias and insane focus on Google to the exclusion of all else.
The NSA is FAR worse than any corporate collaborator, because the NSA has the full weight of the government and law behind it.
A corporation can’t arrest you or put a gun to your head or order you murdered via drone strike.
‘only 300 phone numbers had been used to “initiate” a query’
Well, if those 300 people each called 41 people, and those all called 41 people, and those all called 41 people, we’re already over 20 million.
Holy Crap that's quite a bit of data mining
I worked for a health insurance company. And on occasion we would have to query the raw data for a patient’s prescription history. We only did this when an issue came up or we were performing an audit. Even then there were strict guidelines for how we were to perform a search. This was to protect the privacy of everyone.
For example: If we had a first and last name but no social security number. We were forbidden from doing a search unless we also had their zip code of residence. The searches were to be exact and targeted as possible; which if you care about privacy is how things should be.
This broad untargeted data-mining by the NSA is astounding. Makes me sick.
Re: Holy Crap that's quite a bit of data mining
It’s a bit more like data-strip-mining.
Re: Re: Holy Crap that's quite a bit of data mining
Is data-strip-mining a form of strip search?
Not to nit pick, but the number of queries does not necessarily tell you much. Many people are thinking “select * from phonecalls where country <> ‘US'”, but that’s not how all databases work.
Some data analytics systems will run thousands of queries to satisfy a single request.
They certainly should not be using the total volume of queries to try to minimize the number of abuses, but at this point, they are simply throwing out numbers that are meaningless without context.
We should all be focused on the simple number of times they have abused their power or broken the law. Trust me, the number of times we were not speeding does not matter one bit when it comes to defending the time the radar gun got us.
Re: Re:
By no stretch of the imagination does 20 million queries per month == targeted information gathering. Plain and simple this is indiscriminate data-mining. There is no legitimate argument to the contrary.
Re: Re:
This is a matter of semantics. Are they using queries in the technical sense (and then you would be right while not making their abuse any better) or in the common sense of the word where they do search 20 million times (thus the number of technical queries is way higher) which makes their abuse sound even worse… Now, which definition is the NSA playing by?
Re: Re: Re:
Some other definition that neither common folk nor database administrators would ever dream of.
don't crossreference our statements!
thats our job, we do millions every month and are damn good at it.
The worrying problem is that they have the data available to allow them to make mistakes. This can only increase the pressure to allow other law and security services to access the data increasing the opportunities for mistakes and abuse.
Re: Re:
The NSA does not make mistakes. They simply expand their searches to other relevant data.
Re: Re:
And how long before ICE is using to assist their funders, sorry clients, sorry (what are they?) in the music and movie industries in sending out shakedown, sorry settlement letters?
If the Bacon numbers are a good indicator.
That vastly improves the chances that they’re reading my personal email.
And yours too.
I suspect everyone is few enough degrees from a terrorist suspect to be worthy of inclusion in search queries.
Re: If the Bacon numbers are a good indicator.
Are you calling Kevin Bacon a terrorist?
Well, I suppose all of that anti-establishment dancing could raise some red flags these days.
Re: Re: If the Bacon numbers are a good indicator.
HE IS!
There was a law against dancing, and he violated it and lead others to violate it as well!
So I’m getting between 7 and 8 queries per second, there. Assuming around-the-clock querying.
Between 34 and 35 per second if the querying is only during business hours, where 20 business days defines a “month”.
One query for 3 out of 4 Americans every year
Another way to think of this is the NSA is doing 3 queries for every 4 people in the US.
How is this not illegal searching through illegally seized information.
That’s not a question, it’s a statement. If anyone does have any questions, I refer you to the Constitution, Amendment 4.
Why break the local laws? Should just share.
What I don’t understand is why all the government allies don’t just work together and save themselves some huge embarrassment and political ire. If I had put this together I would have our spies force US companies to hand over our info to the UK spies. The UK spies have their companies giving us their info. We could then both LEGALLY spy on the others’ citizens and share all their findings back with the respective governments legally. Seems so simple.
Everything I needed to know I learned in kindergarten.
In the words of Bonnie Raitt...
Let’s give ’em something to talk about.
The abuses come from LOGGED data, the biggest abuses are in the UNLOGGED queries.
Do you really think there is no skunkworks version of that app that doesn’t log queries? Because I think that’s naive.
Re: Re:
Do you really think there is no skunkworks version of that app that doesn’t log queries? Because I think that’s naive.
I don’t know, the NSA seems to have a compulsive need to keep track of everything. I’m not sure they would be willing to just not log something.
Re: Re: The NSA compulsion to keep track
I don’t know, the NSA seems to have a compulsive need to keep track of everything. I’m not sure they would be willing to just not log something.
I hope you’re right. Thorough records will be convenient when rounding up those responsible for the program.
So Much Open Resistance
It seems incredible at the amount of open resistance to tactics of Security agencies here at Techdirt, given the fact that we are all still speaking English and not some ancient Chinese or Mesopotamian dialect..
Of course they're obviously not lying
LOL.
Of course they’re obviously not lying, and it’s like 10 times that amount…or more…they never lie.
fool me three times…derrr
I remember the data ware house we build 10 years ago had to endure more queries than 20 million a month and those were build for business reporting.