Professor Stephen Gorard s.a.c.gorard@durham.ac.uk
Professor
Do we really need Confidence Intervals in the new statistics?
Gorard, S.
Authors
Abstract
This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as ‘effect’ sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each one is highly predictable from the other. CIs are supposed to be a measure of likelihood or uncertainty in the results, showing a range of possible effect sizes that could have been produced by random sampling variation alone. NNTD is supposed to be a measure of the robustness of the effect size to any variation, including that produced by missing data. Given that they are largely equivalent and interchangeable under the conditions tested here, the paper suggests that both are really measures of robustness. It concludes that NNTD is to be preferred because it requires many fewer assumptions, is more tolerant of missing data, is easier to explain, and directly addresses the key question of whether the underlying effect size is zero or not.
Citation
Gorard, S. (2019). Do we really need Confidence Intervals in the new statistics?. International Journal of Social Research Methodology, 22(3), 281-291. https://doi.org/10.1080/13645579.2018.1525064
Journal Article Type | Article |
---|---|
Acceptance Date | Sep 13, 2018 |
Online Publication Date | Sep 26, 2018 |
Publication Date | 2019 |
Deposit Date | Sep 10, 2018 |
Publicly Available Date | Mar 26, 2020 |
Journal | International Journal of Social Research Methodology |
Print ISSN | 1364-5579 |
Electronic ISSN | 1464-5300 |
Publisher | Taylor and Francis Group |
Peer Reviewed | Peer Reviewed |
Volume | 22 |
Issue | 3 |
Pages | 281-291 |
DOI | https://doi.org/10.1080/13645579.2018.1525064 |
Files
Accepted Journal Article
(629 Kb)
PDF
Copyright Statement
This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Social Research Methodology on 26 Sep 2018, available online: http://www.tandfonline.com/10.1080/13645579.2018.1525064
You might also like
Ethnic proportionality of teachers and students, and the link to school-level outcomes
(2023)
Journal Article
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search