Short communication: Evolution of secondary studies in software engineering

Abstract

Context: Other disciplines commonly employ secondary studies to address the needs of practitioners and policy-makers. Since being adopted by software engineering in 2004, many have been undertaken by researchers. Objective: To assess how the role of secondary studies in software engineering has evolved. Methods: We examined a sample of 131 secondary studies published in a set of five major software engineering journals for the years 2010, 2015 and 2020. These were categorised by their type (e.g. mapping study), their research focus (quantitative/qualitative and practice/methodological), as well as the experience of the first authors. Results: Secondary studies are now a well-established research tool. They are predominantly qualitative and there is extensive use of mapping studies to profile research in particular areas. A significant number are clearly produced as part of postgraduate study, although experienced researchers also conduct many secondary studies. They are sometimes also used as part of a multi-method study. Conclusion: Existing guidelines largely focus upon quantitative systematic reviews. Based on our findings, we suggest that more guidance is needed on how to conduct, analyse, and report qualitative secondary studies.