We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

Do we really need confidence intervals in the new statistics?

Gorard, S. (2019) 'Do we really need confidence intervals in the new statistics?', International journal of social research methodology., 22 (3). pp. 281-291.


This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as ‘effect’ sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each one is highly predictable from the other. CIs are supposed to be a measure of likelihood or uncertainty in the results, showing a range of possible effect sizes that could have been produced by random sampling variation alone. NNTD is supposed to be a measure of the robustness of the effect size to any variation, including that produced by missing data. Given that they are largely equivalent and interchangeable under the conditions tested here, the paper suggests that both are really measures of robustness. It concludes that NNTD is to be preferred because it requires many fewer assumptions, is more tolerant of missing data, is easier to explain, and directly addresses the key question of whether the underlying effect size is zero or not.

Item Type:Article
Full text:(AM) Accepted Manuscript
Download PDF
Publisher Web site:
Publisher statement:This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Social Research Methodology on 26 Sep 2018, available online:
Date accepted:13 September 2018
Date deposited:11 September 2018
Date of first online publication:26 September 2018
Date first made open access:26 March 2020

Save or Share this output

Look up in GoogleScholar