Grossmann, Igor and Rotella, Amanda and Hutcherson, Cendri A. and Sharpinskyi, Konstantyn and Varnum, Michael E.W. and Achter, Sebastian and Dhami, Mandeep K. and Guo, Xinqi Evie and Kara-Yakoubian, Mane and Mandel, David R. and Raes, Louis and Tay, Louis and Vie, Aymeric and Wagner, Lisa and Adamkovic, Matus and Arami, Arash and Arriaga, Patrícia and Bandara, Kasun and Baník, Gabriel and Bartoš, František and Baskin, Ernest and Bergmeir, Christoph and Białek, Michał and Børsting, Caroline K. and Browne, Dillon T. and Caruso, Eugene M. and Chen, Rong and Chie, Bin-Tzong and Chopik, William J. and Collins, Robert N. and Cong, Chin W. and Conway, Lucian G. and Davis, Matthew and Day, Martin V. and Dhaliwal, Nathan A. and Durham, Justin D. and Dziekan, Martyna and Elbaek, Christian T. and Shuman, Eric and Fabrykant, Marharyta and Firat, Mustafa and Fong, Geoffrey T. and Frimer, Jeremy A. and Gallegos, Jonathan M. and Goldberg, Simon B. and Gollwitzer, Anton and Goyal, Julia and Graf-Vlachy, Lorenz and Gronlund, Scott D. and Hafenbrädl, Sebastian and Hartanto, Andree and Hirshberg, Matthew J. and Hornsey, Matthew J. and Howe, Piers D.L. and Izadi, Anoosha and Jaeger, Bastian and Kačmár, Pavol and Kim, Yeun Joon and Krenzler, Ruslan and Lannin, Daniel G. and Lin, Hung-Wen and Lou, Nigel Mantou and Lua, Verity Y.Q. and Lukaszewski, Aaron W. and Ly, Albert L. and Madan, Christopher R. and Maier, Maximilian and Majeed, Nadyanna M. and March, David S. and Marsh, Abigail A. and Misiak, Michal and Myrseth, Kristian Ove R. and Napan, Jaime M. and Nicholas, Jonathan and Nikolopoulos, Konstantinos and O, Jiaqing and Otterbring, Tobias and Paruzel-Czachura, Mariola and Pauer, Shiva and Protzko, John and Raffaelli, Quentin and Ropovik, Ivan and Ross, Robert M. and Roth, Yefim and Røysamb, Espen and Schnabel, Landon and Schütz, Astrid and Seifert, Matthias and Sevincer, A. Timur and Sherman, Garrick T. and Simonsson, Otto and Sung, Ming-Chien and Tai, Chung-Ching and Talhelm, Thomas and Teachman, Bethany A. and Tetlock, Philip E. and Thomakos, Dimitrios and Tse, Dwight C.K. and Twardus, Oliver J. and Tybur, Joshua M. and Ungar, Lyle and Vandermeulen, Daan and Williams, Leighton Vaughan and Vosgerichian, Hrag A. and Wang, Qi and Wang, Ke and Whiting, Mark E. and Wollbrant, Conny E. and Yang, Tao and Yogeeswaran, Kumar and Yoon, Sangsuk and Alves, Ventura r. and Andrews-Hanna, Jessica R. and Bloom, Paul A. and Boyles, Anthony and Charis, Loo and Choi, Mingyeong and Darling-Hammond, Sean and Ferguson, Zoe E. and Kaiser, Cheryl R. and Karg, Simon T. and Ortega, Alberto López and Mahoney, Lori and Marsh, Melvin S. and Martinie, Marcellin F.R.C. and Michaels, Eli K. and Millroth, Philip and Naqvi, Jeanean B. and Ng, Weiting and Rutledge, Robb B. and Slattery, Peter and Smiley, Adam H. and Strijbis, Oliver and Sznycer, Daniel and Tsukayama, Eli and Loon, Austin van and Voelkel, Jan G. and Wienk, Margaux N.A. and Wilkening, Tom and The Forecasting Collaborative, (2022) 'Insights into accuracy of social scientists' forecasts of societal change.', Nature Human Behaviour .
Abstract
How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. Following provision of historical trend data on the domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N=86 teams/359 forecasts), with an opportunity to update forecasts based on new data six months later (Tournament 2; N=120 teams/546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than simple statistical models (historical means, random walk, or linear regressions) or the aggregate forecasts of a sample from the general public (N=802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models, and based predictions on prior data.
| Item Type: | Article |
|---|---|
| Full text: | Publisher-imposed embargo (AM) Accepted Manuscript File format - PDF (3506Kb) |
| Status: | Peer-reviewed |
| Publisher Web site: | https://www.nature.com/nathumbehav/ |
| Date accepted: | 24 November 2022 |
| Date deposited: | 28 November 2022 |
| Date of first online publication: | No date available |
| Date first made open access: | No date available |
Save or Share this output
| Export: | |
| Look up in GoogleScholar |
