Professor Stephen Gorard s.a.c.gorard@durham.ac.uk
Professor
The propagation of errors in experimental data analysis: a comparison of pre- and post-test designs
Gorard, S.
Authors
Abstract
Experimental designs involving the randomization of cases to treatment and control groups are powerful and under-used in many areas of social science and social policy. This paper reminds readers of the pre- and post-test, and the post-test only, designs, before explaining briefly how measurement errors propagate according to error theory. The substance of the paper involves a series of comparisons using the same measurements, all assumed to have a small initial error, and seeing what would happen to that error in the two different experimental designs. The findings from these calculations and simulations are that although post-test only and pre- and post-test designs yield different ‘manifest’ results with the same data, the substantive conclusions drawn would be similar in most real-life situations. However, if these manifest results are assumed to be in error, stemming from small initial errors in the measurements at pre- and post-test, then these substantive conclusions could be completely wrong. In one example, the pre- and post-test designs propagate an initial maximum measurement error of 10% to an error of over 60,000% in the answer. In general, and perhaps counter-intuitively, the post-test only results are less misleading. The paper ends by summarizing the lessons drawn. The key message is that all other things being equal, the post-test only design is to be preferred. We may also need to use bigger samples, and more strictly accurate measures, capable of objective calibration focus on seeking larger effect sizes.
Citation
Gorard, S. (2013). The propagation of errors in experimental data analysis: a comparison of pre- and post-test designs. International Journal of Research & Method in Education, 36(4), 372-385. https://doi.org/10.1080/1743727x.2012.741117
Journal Article Type | Article |
---|---|
Publication Date | Jan 1, 2013 |
Deposit Date | Apr 23, 2013 |
Journal | International Journal of Research & Method in Education |
Print ISSN | 1743-727X |
Electronic ISSN | 1743-7288 |
Publisher | Taylor and Francis Group |
Peer Reviewed | Peer Reviewed |
Volume | 36 |
Issue | 4 |
Pages | 372-385 |
DOI | https://doi.org/10.1080/1743727x.2012.741117 |
Keywords | Research design, Error propagation, Experiments. |
You might also like
Ethnic proportionality of teachers and students, and the link to school-level outcomes
(2023)
Journal Article
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search