Cookies

We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.


Durham Research Online
You are in:

Multisite educational trials: estimating the effect size and its confidence intervals

Singh, A. and Uwimpuhwe, G. and Li, M. and Einbeck, J. and Higgins, S. and Kasim, A. (2022) 'Multisite educational trials: estimating the effect size and its confidence intervals.', International journal of research & method in education., 45 (1). pp. 18-38.

Abstract

In education, multisite trials involve randomisation of pupils into intervention and comparison groups within schools. Most analytical models in multisite educational trials ignore that the impact of an intervention may be school dependent. This study investigates the impact of statistical models on the uncertainty associated with an effect size using comparable outcomes and covariates from ten multisite educational trials funded by the UK’s Education Endowment Foundation. Ordinary least squares (OLS) models often assume that the pupil’s outcomes within schools are independent, which is not always true. Multilevel models address this limitation by incorporating heterogeneity between schools to account for intra-school dependency. This inflates the confidence interval of an effect size obtained from the multilevel models than from an OLS model. For a multisite trial, the heterogeneity between schools also includes the differences in the expected impact of intervention between schools. Ignoring this additional school-by-intervention variation in a multisite trial could affect both its interpretation and conclusions. A robust approach to estimate the confidence intervals for effect size from multisite trials is by treating effect size as a parameter with its distribution. This paper is important for evaluating evidence from multisite trials by accounting for all sources of variability.

Item Type:Article
Full text:(AM) Accepted Manuscript
Download PDF
(995Kb)
Status:Peer-reviewed
Publisher Web site:https://doi.org/10.1080/1743727X.2021.1882416
Publisher statement:This is an Accepted Manuscript of an article published by Taylor & Francis in International journal of research & method in education on 15 February 2021, available online: http://www.tandfonline.com/10.1080/1743727X.2021.1882416
Date accepted:25 November 2020
Date deposited:01 December 2020
Date of first online publication:15 February 2021
Date first made open access:15 August 2022

Save or Share this output

Export:
Export
Look up in GoogleScholar