Big Brother’s New Crystal Ball: Washington Develops Online Data Mining Program to Predict Global Political Unrest

The outrage regarding Washington’s National Security Agency spying, as revealed by whistleblower Edward Snowden, has spread across the world, upsetting diplomatic relations and threatening to shift balances of global power. Yet beyond this spying network lies another, lesser known pilot operation within the NSA called the Intelligence Advanced Research Projects Activity’s Open Source Indicators Program.

This initiative involves academics working at the behest of a research branch of the NSA who are using US government-collected online data to actually predict future events, such as political protests, pandemics, and economic crises—with a focus on Latin America.

This sci-fi type project to create an intelligence “crystal ball” seeks to develop automated analytics programs using open source information such as Facebook, Tweets, Google searches, and other publicly-accessible data in order to stay one step ahead of current events.

“I think citizens in other countries are already worried,” said Robert Albro, Associate Research Professor at American University’s Center for Latin American and Latino Studies. “While, so far as I’m aware, the details of the IARPA’s Open Source Indicators Program are not widely-known in Latin America at present, programs of this sort will be understood in the political and diplomatic context of the recent post-Snowden revelations about NSA cyber-espionage in the region, particularly with respect to both Mexico and Brazil.”

At a recent gathering at the United Nations, a united group of Latin American presidents confronted the Obama administration about its spying operations in the region, denouncing this affront to regional sovereignty. But the new IARPA program extends these unpopular spying efforts in a new direction.

“The goal that IARPA has is to eventually transition this to the intelligence community, and use it for something like the National Intelligence Estimates,” Jenn Carter, who works on the project, told the BBC last year. National Intelligence Estimates are US government intelligence reports on national security issues that predict future events.

One so-called success story of the new Open Source Indicators Program, as reported in The Wall Street Journal on September 10th by Rachel King, involves a team of university academics and representatives from the private sector which “forecasted several protests in Brazil” for the South American nation’s independence day. The same team, writes King, also successfully forecasted protests in Paraguay after the 2012 coup against President Fernando Lugo.

Albro was dubious of the program’s revelations on the Brazil protests, calling it a “drinking the kool-aid moment,” where the project’s supporters will use this alleged success to reinforce their beliefs about the project’s viability. 

David Price, a Professor of Anthropology in the Department of Social Justice at Saint Martin’s University shares a similar sentiment. “I am extremely skeptical that these sort of planes of prediction will work out the way they want them to,” he said. “However, I understand why those in power who want to know what’s going on in the world, and want to manipulate it, are so interested in this.”

The US intelligence community has come under fire over the last dozen years for failures to predict major events, from 9/11 and the Iraq War to the Arab Spring. One notable critique emerged in a February 2011 Senate hearing, when Dianne Feinstein (D-Calif.) said that, in regards to the upheaval in Egypt, “she was particularly concerned that the CIA and other agencies had ignored open-source intelligence on the protests, a reference to posts on Facebook and other publicly accessible Web sites used by organizers of the protests against the Mubarak government.”

If possible, IARPA’s program seeks to fix that. “Our focus is to beat the news with greater accuracy and to do it faster by combining [various sets of] data, and we are seeing that it is possible,” Jason Matheny, program manager of the Open Source Indicators Program, told Bloomberg Business Week in February of this year.

The privacy of the users whose information is being culled for this predictive analytics program could be a controversial issue. Naren Ramakrishnan, a Professor of Engineering at Virginia Tech who heads up the pilot NSA program told the Wall Street Journal in June that his research team was trying to protect the privacy of people even while they collected this data. “In the case of civil unrest, we haven’t come to the point of modeling government opposition groups,” he said. Ramakrishnan explained that they seek to predict when a protest might happen, but not who would be participating in it.

But after Snowden’s leaks regarding Washington’s indiscriminate spying, many experts have raised concerns that this program will violate personal rights and privacy as well.

American University’s Albro said that a “viable, ethical framework on how to collect data from social media” and protect privacy still needs to be developed and articulated for research purposes in general.

“We should all be worried,” said anthropology Professor David Price, also the author of Weaponizing Anthropology: Social Science in Service of the Militarized State. “It’s going to be used domestically, I imagine, as much as it is going to be used internationally.”

Price, a critic of academics’ collaboration with the Pentagon for its Human Terrain System in Iraq and Afghanistan, which involves social scientists providing the military with information on the local people and culture where troops are deployed, none-the-less thinks open source intelligence can potentially be very useful in preventing disasters.

However, he explained, “I don’t see them wanting to predict and prevent things like famine, but rather they want to figure out ways to support regimes which support American interests as part of a larger political program we should all be worried about.”

Cyril Mychalejko is an editor at, an online magazine covering politics and activism in Latin America.