Singh, A. and Uwimpuhwe, G. and Li, M. and Einbeck, J. and Higgins, S. and Kasim, A. (2022) 'Multisite educational trials: estimating the effect size and its confidence intervals.', International journal of research & method in education., 45 (1). pp. 18-38.
In education, multisite trials involve randomisation of pupils into intervention and comparison groups within schools. Most analytical models in multisite educational trials ignore that the impact of an intervention may be school dependent. This study investigates the impact of statistical models on the uncertainty associated with an effect size using comparable outcomes and covariates from ten multisite educational trials funded by the UK’s Education Endowment Foundation. Ordinary least squares (OLS) models often assume that the pupil’s outcomes within schools are independent, which is not always true. Multilevel models address this limitation by incorporating heterogeneity between schools to account for intra-school dependency. This inflates the confidence interval of an effect size obtained from the multilevel models than from an OLS model. For a multisite trial, the heterogeneity between schools also includes the differences in the expected impact of intervention between schools. Ignoring this additional school-by-intervention variation in a multisite trial could affect both its interpretation and conclusions. A robust approach to estimate the confidence intervals for effect size from multisite trials is by treating effect size as a parameter with its distribution. This paper is important for evaluating evidence from multisite trials by accounting for all sources of variability.
|Full text:||(AM) Accepted Manuscript|
Download PDF (995Kb)
|Publisher Web site:||https://doi.org/10.1080/1743727X.2021.1882416|
|Publisher statement:||This is an Accepted Manuscript of an article published by Taylor & Francis in International journal of research & method in education on 15 February 2021, available online: http://www.tandfonline.com/10.1080/1743727X.2021.1882416|
|Date accepted:||25 November 2020|
|Date deposited:||01 December 2020|
|Date of first online publication:||15 February 2021|
|Date first made open access:||15 August 2022|
Save or Share this output
|Look up in GoogleScholar|