We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

An empirical unravelling of Lord’s Paradox.

Xiao, Z. and Higgins, S. and Kasim, A. (2017) 'An empirical unravelling of Lord’s Paradox.', Journal of experimental education. .


Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size estimates from 34 large-scale educational interventions and found that impact estimates are affected in magnitude, with or without reversal in sign, when there is substantial baseline imbalance. We also demonstrated that multilevel modeling can ameliorate the divergence in sign and/or magnitude of effect estimation, which, together with project specific knowledge, promises to help those who are presented with conflicting or confusing evidence in decision-making.

Item Type:Article
Keywords:Lord's Paradox, RCT, Multilevel Modelling, Evidence-Based Policy, Evaluation.
Full text:Publisher-imposed embargo
(AM) Accepted Manuscript
First Live Deposit - 29 August 2017
File format - PDF
Full text:Publisher-imposed embargo until 07 May 2019.
(AM) Accepted Manuscript
File format - PDF (Revised version)
Publisher Web site:
Publisher statement:This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of Experimental Education on 07 Nov 2017, available online:
Record Created:29 Aug 2017 15:13
Last Modified:23 Nov 2017 14:21

Social bookmarking: del.icio.usConnoteaBibSonomyCiteULikeFacebookTwitterExport: EndNote, Zotero | BibTex
Look up in GoogleScholar | Find in a UK Library