Gorard, S. (2019) 'Do we really need confidence intervals in the new statistics?', International journal of social research methodology., 22 (3). pp. 281-291.
This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as ‘effect’ sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each one is highly predictable from the other. CIs are supposed to be a measure of likelihood or uncertainty in the results, showing a range of possible effect sizes that could have been produced by random sampling variation alone. NNTD is supposed to be a measure of the robustness of the effect size to any variation, including that produced by missing data. Given that they are largely equivalent and interchangeable under the conditions tested here, the paper suggests that both are really measures of robustness. It concludes that NNTD is to be preferred because it requires many fewer assumptions, is more tolerant of missing data, is easier to explain, and directly addresses the key question of whether the underlying effect size is zero or not.
|Full text:||(AM) Accepted Manuscript|
Download PDF (614Kb)
|Publisher Web site:||https://doi.org/10.1080/13645579.2018.1525064|
|Publisher statement:||This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Social Research Methodology on 26 Sep 2018, available online: http://www.tandfonline.com/10.1080/13645579.2018.1525064|
|Date accepted:||13 September 2018|
|Date deposited:||11 September 2018|
|Date of first online publication:||26 September 2018|
|Date first made open access:||26 March 2020|
Save or Share this output
|Look up in GoogleScholar|