Polling The (Chat)Room
If two minds are better than one, how much better are two thousand minds? A new hive mind project known as Metaculus is attempting to find out.
Essentially the crowdsourcing of knowledge, the concept of "hive mind," or collective intelligence, has been explored within the context of such fields as biology, robotics, and political science. Now, through their online community Metaculus, physicists Anthony Aguirre (University of California), Gregory Laughlin (Yale), and former postdoc Max Wainwright are testing the accuracy of hive mind in making predictions about the future of such fields as science, technology, and space.
So, how can humans help predict the future?
After signing up for a free account, Metaculus users answer yes-or-no questions such as "Will SpaceX launch for Mars in the 2018 window?" or "Will a self-driving car be available for ride service by the end of 2018?" Users also indicate their level of certainty in their answer on a scale of 0 to 100. Each question has a resolution date, and users earn points for every correct prediction they make. Participants are ranked on the site according to their accuracy score, adding a competitive gaming element to the project
After questions are resolved, the accuracy of the "hive" can be charted so the site's creators can determine whether the collective instincts of the group were able to predict the events' outcomes more accurately than individuals within the group could. So far, the answer seems to be "Yes."
The hive mind works.
"Individuals outperform the community predictions on lots of individual questions, but in aggregate, you are better off trusting the community than nearly any individual," Aguirre noted in a recent email exchange. "The aggregate predictions (by any aggregation method and including all predictors) are substantially more precise and well-calibrated than chance, [and] Metaculus users do quite a good job at making well-calibrated predictions, especially for low probabilities."
Notably, these early Metaculus findings are in line with those of other hive-mind projects.
The Intelligence Advanced Research Projects Activity's (IARPA) Good Judgment Project, a four-year research study launched in 2012 as part of a government-sponsored forecasting tournament, has solicited predictions on global events from thousands of people from across the world. They report that the collective forecasts they've received have been "surprisingly accurate," while UNU, a web-based hive mind, has correctly predicted the winners of several sporting events, political debates, and awards shows.
A Strong Hive is Made of Quality Bees
In addition to offering further proof of the value of hive-mind research, another notable finding from both Metaculus and the Good Judgment Project is the emergence of a very small number of "superpredictors," individuals who consistently make accurate predictions.
Aguirre notes that there are a "handful" of predictors whose average Brier score is lower than that of the community on the same set of questions, and in an interview with CHIPS, experts from the Good Judgment Project cited the emergence of consistently accurate individuals within their project as "evidence that forecasting is a skill that can be learned and that forecasters who achieve outstanding results in one year tend to be among the top performers in subsequent years."
Moreover, as noted by Science News, Metaculus could have a tertiary benefit as a way to determine public interest in future academic research as well. Organizations that fund research on the topics covered by Metaculus could view the site as a sounding board of sorts, seeing which types of questions receive the most votes and gauging the public perception of those topics.
With value on so many levels, Metaculus has already proven to be a worthwhile undertaking. As its user base continues to grow and more predications are made, this hive mind project will provide even more insight into the predictive power of the collective.
Statements in this article have been slightly edited for clarity and brevity.
Share This Article