Skip to main content

Research Repository

Advanced Search

Insights into accuracy of social scientists' forecasts of societal change

Grossmann, Igor; Rotella, Amanda; Hutcherson, Cendri A.; Sharpinskyi, Konstantyn; Varnum, Michael E.W.; Achter, Sebastian; Dhami, Mandeep K.; Guo, Xinqi Evie; Kara-Yakoubian, Mane; Mandel, David R.; Raes, Louis; Tay, Louis; Vie, Aymeric; Wagner, Lisa; Adamkovic, Matus; Arami, Arash; Arriaga, Patrícia; Bandara, Kasun; Baník, Gabriel; Bartoš, František; Baskin, Ernest; Bergmeir, Christoph; Białek, Michał; Børsting, Caroline K.; Browne, Dillon T.; Caruso, Eugene M.; Chen, Rong; Chie, Bin-Tzong; Chopik, William J.; Collins, Robert N.; Cong, Chin W.; Conway, Lucian G.; Davis, Matthew; Day, Martin V.; Dhaliwal, Nathan A.; Durham, Justin D.; Dziekan, Martyna; Elbaek, Christian T.; Shuman, Eric; Fabrykant, Marharyta; Firat, Mustafa; Fong, Geoffrey T.; Frimer, Jeremy A.; Gallegos, Jonathan M.; Goldberg, Simon B.; Gollwitzer, Anton; Goyal, Julia; Graf-Vlachy, Lorenz; Gronlund, Scott D.; Hafenbrädl, Sebastian; Hartanto, Andree; Hirshberg, Matthew J.; Hornsey, Matthew J.; Howe, Piers D.L.; Izadi,...

Insights into accuracy of social scientists' forecasts of societal change Thumbnail


Authors

Igor Grossmann

Amanda Rotella

Cendri A. Hutcherson

Konstantyn Sharpinskyi

Michael E.W. Varnum

Sebastian Achter

Mandeep K. Dhami

Xinqi Evie Guo

Mane Kara-Yakoubian

David R. Mandel

Louis Raes

Louis Tay

Aymeric Vie

Lisa Wagner

Matus Adamkovic

Arash Arami

Patrícia Arriaga

Kasun Bandara

Gabriel Baník

František Bartoš

Ernest Baskin

Christoph Bergmeir

Michał Białek

Caroline K. Børsting

Dillon T. Browne

Eugene M. Caruso

Rong Chen

Bin-Tzong Chie

William J. Chopik

Robert N. Collins

Chin W. Cong

Lucian G. Conway

Matthew Davis

Martin V. Day

Nathan A. Dhaliwal

Justin D. Durham

Martyna Dziekan

Christian T. Elbaek

Eric Shuman

Marharyta Fabrykant

Mustafa Firat

Geoffrey T. Fong

Jeremy A. Frimer

Jonathan M. Gallegos

Simon B. Goldberg

Anton Gollwitzer

Julia Goyal

Lorenz Graf-Vlachy

Scott D. Gronlund

Sebastian Hafenbrädl

Andree Hartanto

Matthew J. Hirshberg

Matthew J. Hornsey

Piers D.L. Howe

Anoosha Izadi

Bastian Jaeger

Pavol Kačmár

Yeun Joon Kim

Ruslan Krenzler

Daniel G. Lannin

Hung-Wen Lin

Nigel Mantou Lou

Verity Y.Q. Lua

Aaron W. Lukaszewski

Albert L. Ly

Christopher R. Madan

Maximilian Maier

Nadyanna M. Majeed

David S. March

Abigail A. Marsh

Michal Misiak

Kristian Ove R. Myrseth

Jaime M. Napan

Jonathan Nicholas

Jiaqing O

Tobias Otterbring

Mariola Paruzel-Czachura

Shiva Pauer

John Protzko

Quentin Raffaelli

Ivan Ropovik

Robert M. Ross

Yefim Roth

Espen Røysamb

Landon Schnabel

Astrid Schütz

Matthias Seifert

A. Timur Sevincer

Garrick T. Sherman

Otto Simonsson

Ming-Chien Sung

Chung-Ching Tai

Thomas Talhelm

Bethany A. Teachman

Philip E. Tetlock

Dimitrios Thomakos

Dwight C.K. Tse

Oliver J. Twardus

Joshua M. Tybur

Lyle Ungar

Daan Vandermeulen

Leighton Vaughan Williams

Hrag A. Vosgerichian

Qi Wang

Ke Wang

Mark E. Whiting

Conny E. Wollbrant

Tao Yang

Kumar Yogeeswaran

Sangsuk Yoon

Ventura r Alves

Jessica R. Andrews-Hanna

Paul A. Bloom

Anthony Boyles

Loo Charis

Mingyeong Choi

Sean Darling-Hammond

Zoe E. Ferguson

Cheryl R. Kaiser

Simon T. Karg

Alberto López Ortega

Lori Mahoney

Melvin S. Marsh

Marcellin F.R.C. Martinie

Eli K. Michaels

Philip Millroth

Jeanean B. Naqvi

Weiting Ng

Robb B. Rutledge

Peter Slattery

Adam H. Smiley

Oliver Strijbis

Daniel Sznycer

Eli Tsukayama

Austin van Loon

Jan G. Voelkel

Margaux N.A. Wienk

Tom Wilkening

The Forecasting Collaborative



Abstract

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. Following provision of historical trend data on the domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N=86 teams/359 forecasts), with an opportunity to update forecasts based on new data six months later (Tournament 2; N=120 teams/546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than simple statistical models (historical means, random walk, or linear regressions) or the aggregate forecasts of a sample from the general public (N=802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models, and based predictions on prior data.

Citation

Grossmann, I., Rotella, A., Hutcherson, C. A., Sharpinskyi, K., Varnum, M. E., Achter, S., …Collaborative, T. F. (2023). Insights into accuracy of social scientists' forecasts of societal change. Nature Human Behaviour, 7(4), 484-501. https://doi.org/10.1038/s41562-022-01517-1

Journal Article Type Article
Acceptance Date Nov 24, 2022
Online Publication Date Feb 9, 2023
Publication Date 2023-04
Deposit Date Nov 28, 2022
Publicly Available Date Aug 10, 2023
Journal Nature Human Behaviour
Publisher Nature Research
Peer Reviewed Peer Reviewed
Volume 7
Issue 4
Pages 484-501
DOI https://doi.org/10.1038/s41562-022-01517-1
Public URL https://durham-repository.worktribe.com/output/1187810

Files





You might also like



Downloadable Citations