Secretive agency uses AI, human “forecasters” to predict the future

A US government intelligence agency develops cutting-edge tech to predict future events.

IARPA, a research arm of the U.S. government intelligence community, is focused on predicting the future. The organization uses teams of human non-experts and AI machine learning to forecast events. IARPA also conducts advanced research in numerous other fields, funding rotating programs.

As far as secretive government projects go, the objectives of IARPA may be the riskiest and most far-reaching. With its mission to foster “high-risk, high-payoff” programs, this research arm of the U.S. intelligence community literally tries to predict the future. Staffed by spies and PhDs, this organization aims to provide decision-makers with real, accurate predictions of geopolitical events, using AI and human “forecasters”.

IARPA, which stands for Intelligence Advanced Research Projects Activity, was founded in 2006 as part of the Office of the Director of National Intelligence. Some of the projects that it’s funded focused on advancements in quantum computing, cryogenic computing, face recognition, universal language translators, and other projects that would fit well in a Hollywood action movie plot. But perhaps its main goal is to produce “anticipatory intelligence”. It’s a spy agency, after all.

In the interest of national security, IARPA wants to identify major world events before they happen. To that end, it is also in the business of looking for terrorists, hackers, or any perceived enemies of the United States. Wouldn’t you rather stop a crime before it happens?

Of course, that’s when we get into tricky political and sci-fi territory. Much of the research done by IARPA is actually out in the open, utilizing the public and experts in advancing technologies. It is available for “open solicitations,” forecasting tournaments, and has prize challenges for the public. You can pretty much send your idea in right now. But what happens to the R&D once it leaves the lab, is, of course, often for only the NSA and the CIA to know.

The National Security Agency expert James Bradford wrote that the agency is ultimately looking to create a system where huge amounts of data about people’s lives would be mined in real-time, for the purpose of preventing actions detrimental to the nation. In his article of the Pittsburgh Post-Gazette, Bradford described IARPA’s goal is to create very powerful automated computer systems, managed through artificial intelligence, which would be “capable of cataloging the lives of everyone everywhere, 24/ 7.” Such programs would be able to instantaneously access data streams belonging to citizens, whether from social media or anywhere else. Being able to analyze “every Facebook post, tweet, and YouTube video; every tollbooth tag number; every GPS download, web search, and news feed; every street camera video; every restaurant reservation on Open Table — largely eliminates surprise from the intelligence equation,” as writes Bradford.

Of course, one would suspect much of this is going on already. IARPA’s Mercury program, for example, concentrates on data mining millions of private communications that are gathered by the National Security Agency. While it can certainly be argued that such a program is a national security necessity, working to spot terrorists and elements that can lead to social unrest, the potential for misuse and infringement on privacy rights has certainly alerted observers.

A fascinating recent project funded by IARPA is called SAGE, which stands for Synergistic Anticipation of Geopolitical Events. As you may expect from such a lofty title, the researchers involved in this endeavor a looking to predict the future. What’s interesting, this project is aimed at tapping non-experts – humans who would use AI machine learning to make qualified statements about what will happen.

Led by Aram Galstyan, director of the Artificial Intelligence Division at the USC Viterbi Information Sciences Institute (ISI), the project has actually been successful in making concrete predictions, like knowing when North Korea would launch its missile tests. SAGE works by utilizing large sets of human non-expert predictors, pooling their powers by working together, making them “more accurate and faster than a single human subject expert,” as explains a USC press release. However, the information these humans or “forecasters” use to make predictions is gathered through a variety of machine learning technologies.

The topics looked at by the predictors include such questions as “Will any G7 nation engage in an acknowledged national military attack against Syria [by a given date]?” Or they might want to figure out the exact number for how much oil Venezuela might produce in a specific month.

Leaders among the forecasters, who make the most accurate predictions, are ranked and highlighted with badges.

This AI-assisted crowdsourced Nostradamus has worked out quite well, according to Fred Morstatter, a USC computer scientist. “We believe that’s the case because the numbers we’re seeing indicate we are outpacing a system that uses only humans,” he remarked.

SAGE’s hybrid model functions by offering humans information derived by the machine in charts that show trends, along with specific predictions by the AI. “SAGE works because humans have one side of the coin, and machines have the other side,” pointed Morstatter. And on yet another side you would have the National Intelligence apparatus.

Do you have a good idea for future-oriented national security research? You can actually apply to be a IARPA program manager. Current managers, who rotate every 3 to 5 years, are working on a vast variety of fields, like forecasting, linguistics, underwater technology, aerospace propulsion, atomic physics, artificial intelligence, biometrics, neuroscience, optics and more. Check out the list of existing programs.

BigThink