data-mining

Data mining for the masses: Google merges Trends, Insights

Quick show of hands: Who knew that West Virginia leads the U.S. when it comes to searches for spooky things? Or that the interest in the topic line " red hot chili peppers" waxed and waned in different regions of Europe this summer when the band by the same name toured there?

Those are two of the data points Google said you'd be able to glean in conjunction with its merger of Trends and Insights for Search into a new product containing features from both products.

The changes, which Google says will make it easier to analyze … Read more

Data mining's adult challenges

Probably no data-mining legend has been more pervasive than the "beer and diapers" story, which apparently dates back to an early 1990s project that data-warehousing pioneer Teradata (then part of NCR) conducted for the Osco Drug retail chain.

As the story goes, they discovered that beer and diapers frequently appeared together in a shopping basket on certain days; the presumed explanation was that fathers picking up diapers bought a six-pack when they were out anyway. This correlation was then used to optimize displays and pricing in the stores.

That's the story anyway. The reality, as best anyone can determine, … Read more

nPario strip-mines online usage data for more effective ads

Last week, Bassel Ojjeh, CEO of the data analytics startup nPario, gave me an interesting look into how behavior tracking is being used in advertsing, as well as a somewhat startling look at his previous employer, Yahoo.

Prior to starting nPario, Ojjeh ran Yahoo's Strategic Data Solutions group, which he says managed one of the largest data stores in the world. When he was there in 2009, he said, it collected over 8 petabytes of usage data from 550 million users, over about 16,000 servers. This data was used to improve Yahoo's user experience, advertising targeting, and … Read more

Social networks--the new front in war on terror

Unnamed intelligence agencies and certain academics have yet to give up on data mining to identify terrorists and predict attacks, despite a 352-page tome published last year pronouncing the practice a waste of time.

The U.S. is spending "hundreds of millions of dollars" to develop techniques to mine the mountains of information gleaned from e-mails, telephone calls, interviews with suspects, and now social networks to build-up Facebook-style databanks on international terrorists, according to a recent piece in the British newspaper, The Independent.

The result has been the arrest and interrogation of "many thousands of innocent people&… Read more

Panel: Government data-mining programs lack oversight

Americans leave behind countless digital footprints from everyday activities like making a phone call or using a credit card--footprints government agencies regularly track as part of their counterterrorism efforts.

The collection, retention, and dissemination of this information has dangerously escaped public oversight and congressional scrutiny, public sector experts warned Congress on Wednesday. If the next Congress and administration do not take steps to rein in these programs that are bloating the federal government, they said, it will come at the expense of both civil liberties and national security.

Policy experts laid out their concerns to the House Homeland Security Committee … Read more

We Feel Fine

We Feel Fine is “an exploration of human emotion on a global scale.” The site, created by Brooklyn-based artist Jonathan Harris and Stanford computational math professor and former Google employee Sep Kamvar, looks like exactly the result of these two minds combined: emotional data mining with a human touch and an artistic interface -- a particularly beautiful application of moodgraphics.

The site is driven by a huge database that browses the web for emotional expressions around the globe and maps them graphically: “Every few minutes, the system searches the world’s newly posted blog entries for occurrences of the phrases ‘… Read more

Maltego and the science of 'open-source' snooping

Data mining used to be an expensive, somewhat esoteric affair. But as Forbes highlights in a recent article on Maltego, a new "open-source intelligence" tool, new technology "lets just about anybody do the kind of data mining that in the past only fraud investors, government specialists, and hackers typically could do."

Should we be worried?

Not really. Maltego doesn't snoop into closed data repositories, but instead mines publicly available data and helps to make inferences and connections between the disparate data sets. Here are two examples:

Worried about information leaks your company? Input lists of … Read more

Buzz Out Loud 826: Introducing the Jabra Weimaraner

That's the dog-sized Bluetooth headset you can buy to go with today's real news item, the new BlackBerry Storm: announced but not released. Until it comes out, though, it's basking in the warm glow of the "meeting and maybe exceeding expectations" judgments coming from the media. In sum, it sounds cool. And Google decides it's high time they made some money on that whole YouTube thing.

Listen now: Download today's podcast EPISODE 826

BlackBerry Storm 9500 hands-on http://www.engadget.com/2008/10/08/blackberry-storm-9500-hands-on/ http://news.cnet.com/8301-17938_105-10059498-1.html

YouTube adds … Read more

Teradata jumps on SAP acquisition speculation

Updated at 2:40 p.m. PDT, with SAP and SAS declining to comment.

Shares of data warehouse management company Teradata jumped nearly 7 percent in trading Tuesday, following an analyst report that the company would make a good fit with enterprise software applications behemoth SAP.

And while JMP Securities analyst Pat Walravens, notes that SAP has "hinted in recent weeks" it may be considering another acquisition and he believes Teradata would make a good fit, SAP sources say no such transaction is in the immediate works.

Teradata, which has a market capitalization of $3.94 billion, would … Read more