Skip to main content

Research Repository

Advanced Search

The contribution that empirical studies performed in industry make to the findings of systematic reviews: A tertiary study

Budgen, David; Brereton, Pearl; Williams, Nikki; Drummond, Sarah

The contribution that empirical studies performed in industry make to the findings of systematic reviews: A tertiary study Thumbnail


Authors

Pearl Brereton

Nikki Williams

Sarah Drummond



Abstract

Context: Systematic reviews can provide useful knowledge for software engineering practice, by aggregating and synthesising empirical studies related to a specific topic. Objective: We sought to assess how far the findings of systematic reviews addressing practice-oriented topics have been derived from empirical studies that were performed in industry or that used industry data. Method: We drew upon and augmented the data obtained from a tertiary study that performed a systematic review of systematic reviews published in the period up to the end of 2015, seeking to identify those with findings that are relevant for teaching and practice. For the supplementary analysis reported here, we then examined the profiles of the primary studies as reported in each systematic review. Results: We identified 48 systematic reviews as candidates for further analysis. The many differences that arise between systematic reviews, together with the incompleteness of reporting for these, mean that our counts should be treated as indicative rather than definitive. However, even when allowing for problems of classification, the findings from the majority of these systematic reviews were predominantly derived from using primary studies conducted in industry. There was also an emphasis upon the use of case studies, and a number of the systematic reviews also made some use of weaker ‘experience’ or even ‘opinion’ papers. Conclusions: Primary studies from industry play an important role as inputs to systematic reviews. Using more rigorous industry-based primary studies can give greater authority to the findings of the systematic reviews, and should help with the creation of a corpus of sound empirical data to support evidence-informed decisions.

Citation

Budgen, D., Brereton, P., Williams, N., & Drummond, S. (2018). The contribution that empirical studies performed in industry make to the findings of systematic reviews: A tertiary study. Information and Software Technology, 94, 234-244. https://doi.org/10.1016/j.infsof.2017.10.012

Journal Article Type Article
Acceptance Date Oct 16, 2017
Online Publication Date Oct 23, 2017
Publication Date Feb 1, 2018
Deposit Date Oct 17, 2017
Publicly Available Date Mar 29, 2024
Journal Information and Software Technology
Print ISSN 0950-5849
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 94
Pages 234-244
DOI https://doi.org/10.1016/j.infsof.2017.10.012
Keywords Systematic review, Primary study, Industry study, Case study.

Files




You might also like



Downloadable Citations