Cookies

We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.


Durham Research Online
You are in:

The MIDAS touch: accurate and scalable missing-data imputation with deep learning.

Lall, Ranjit and Robinson, Thomas (2022) 'The MIDAS touch: accurate and scalable missing-data imputation with deep learning.', Political analysis., 30 (2). pp. 179-196.

Abstract

Principled methods for analyzing missing values, based chiefly on multiple imputation, have become increasingly popular yet can struggle to handle the kinds of large and complex data that are also becoming common. We propose an accurate, fast, and scalable approach to multiple imputation, which we call MIDAS (Multiple Imputation with Denoising Autoencoders). MIDAS employs a class of unsupervised neural networks known as denoising autoencoders, which are designed to reduce dimensionality by corrupting and attempting to reconstruct a subset of data. We repurpose denoising autoencoders for multiple imputation by treating missing values as an additional portion of corrupted data and drawing imputations from a model trained to minimize the reconstruction error on the originally observed portion. Systematic tests on simulated as well as real social science data, together with an applied example involving a large-scale electoral survey, illustrate MIDAS’s accuracy and efficiency across a range of settings. We provide open-source software for implementing MIDAS.

Item Type:Article
Full text:(AM) Accepted Manuscript
Available under License - Creative Commons Attribution No Derivatives 4.0.
Download PDF
(2343Kb)
Status:Peer-reviewed
Publisher Web site:https://doi.org/10.1017/pan.2020.49
Date accepted:08 October 2020
Date deposited:14 December 2020
Date of first online publication:26 February 2021
Date first made open access:14 December 2020

Save or Share this output

Export:
Export
Look up in GoogleScholar