We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

The misreporting trade-off between list experiments and direct questions in practice: Partition validation evidence from two countries

Kuhn, Patrick M. and Vivyan, Nick (2021) 'The misreporting trade-off between list experiments and direct questions in practice: Partition validation evidence from two countries.', Political analysis. .


To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase non-strategic misreporting. We provide the first empirical assessment of this trade-off between strategic and non-strategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents' true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of non-strategic reporting errors. For both list experiments, partition validation reveals non-strategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.

Item Type:Article
Full text:Publisher-imposed embargo
(AM) Accepted Manuscript
Available under License - Creative Commons Attribution Non-commercial No Derivatives 4.0.
File format - PDF
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution 4.0.
Download PDF (Advance online version)
Publisher Web site:
Publisher statement:© The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence ( licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Supplementary material:
Date accepted:22 December 2020
Date deposited:06 January 2021
Date of first online publication:16 April 2021
Date first made open access:21 January 2021

Save or Share this output

Look up in GoogleScholar