How Reliable Are Systematic Reviews in Empirical Software Engineering?

  • MacDonell S
  • Shepperd M
  • Kitchenham B
 et al. 
  • 3


    Mendeley users who have this article in their library.
  • N/A


    Citations of this article.


BACKGROUND-The systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE-The aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular, we wish to investigate the consistency of process and the stability of outcomes. METHOD-We compare the results of two independent reviews undertaken with a common research question. RESULTS-The two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS-In addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method.

Author-supplied keywords

  • cost estimation
  • empirical software engineering
  • meta analysis
  • systematic review

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

There are no full text links


  • S G MacDonell

  • M J Shepperd

  • B A Kitchenham

  • E Mendes

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free