The power of noise and the art of prediction

Xiao, Z.; Higgins, S.

doi:10.1016/j.ijer.2017.10.006

The power of noise and the art of prediction

Xiao, Z.; Higgins, S.

Authors

Z. Xiao

Professor Steven Higgins s.e.higgins@durham.ac.uk
Professor

Abstract

Data analysis usually aims to identify a particular signal, such as an intervention effect. Conventional analyses often assume a specific data generation process, which implies a theoretical model that best fits the data. Machine learning techniques do not make such an assumption. In fact, they encourage multiple models to compete on the same data. Applying logistic regression and machine learning algorithms to real and simulated datasets with different features of noise and signal, we demonstrate that no single model dominates others under all circumstances. By showing when different models shine or struggle, we argue that it is important to conduct predictive analyses using cross-validation for better evidence that informs decision making.

Citation

Xiao, Z., & Higgins, S. (2017). The power of noise and the art of prediction. International Journal of Educational Research, 87, 36-46. https://doi.org/10.1016/j.ijer.2017.10.006

Journal Article Type	Article
Acceptance Date	Oct 25, 2017
Online Publication Date	Dec 1, 2017
Publication Date	Dec 1, 2017
Deposit Date	Apr 25, 2018
Publicly Available Date	Jun 1, 2019
Journal	International Journal of Educational Research
Print ISSN	0883-0355
Publisher	Elsevier
Peer Reviewed	Peer Reviewed
Volume	87
Pages	36-46
DOI	https://doi.org/10.1016/j.ijer.2017.10.006

Files

Accepted Journal Article (10.4 Mb)
PDF

Copyright Statement
© 2018 This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/