Skip to main content

Research Repository

Advanced Search

Multisite educational trials: estimating the effect size and its confidence intervals

Singh, A.; Uwimpuhwe, G.; Li, M.; Einbeck, J.; Higgins, S.; Kasim, A.

Multisite educational trials: estimating the effect size and its confidence intervals Thumbnail


Authors

M. Li

A. Kasim



Abstract

In education, multisite trials involve randomisation of pupils into intervention and comparison groups within schools. Most analytical models in multisite educational trials ignore that the impact of an intervention may be school dependent. This study investigates the impact of statistical models on the uncertainty associated with an effect size using comparable outcomes and covariates from ten multisite educational trials funded by the UK’s Education Endowment Foundation. Ordinary least squares (OLS) models often assume that the pupil’s outcomes within schools are independent, which is not always true. Multilevel models address this limitation by incorporating heterogeneity between schools to account for intra-school dependency. This inflates the confidence interval of an effect size obtained from the multilevel models than from an OLS model. For a multisite trial, the heterogeneity between schools also includes the differences in the expected impact of intervention between schools. Ignoring this additional school-by-intervention variation in a multisite trial could affect both its interpretation and conclusions. A robust approach to estimate the confidence intervals for effect size from multisite trials is by treating effect size as a parameter with its distribution. This paper is important for evaluating evidence from multisite trials by accounting for all sources of variability.

Citation

Singh, A., Uwimpuhwe, G., Li, M., Einbeck, J., Higgins, S., & Kasim, A. (2022). Multisite educational trials: estimating the effect size and its confidence intervals. International Journal of Research & Method in Education, 45(1), 18-38. https://doi.org/10.1080/1743727x.2021.1882416

Journal Article Type Article
Acceptance Date Nov 25, 2020
Online Publication Date Feb 15, 2021
Publication Date 2022
Deposit Date Nov 30, 2020
Publicly Available Date Mar 29, 2024
Journal International Journal of Research & Method in Education
Print ISSN 1743-727X
Electronic ISSN 1743-7288
Publisher Taylor and Francis Group
Peer Reviewed Peer Reviewed
Volume 45
Issue 1
Pages 18-38
DOI https://doi.org/10.1080/1743727x.2021.1882416

Files





You might also like



Downloadable Citations