This paper argues that the common practice of benchmarking is inadequate as a scientific evaluation methodology. It further attempts to introduce the empirical tradition of the physical sciences by using techniques from Statistical Design of Experiments applied to the example of SPARQL endpoint performance evaluation. It does so by studying full as well as fractional factorial experiments designed to evaluate an assertion that some change introduced in a system has improved performance. This paper does not present a finished experimental design, rather its main focus is didactical, to shift the focus of the community away from benchmarking towards higher scientific rigor. © 2013 Springer-Verlag.
CITATION STYLE
Kjernsmo, K., & Tyssedal, J. S. (2013). Introducing statistical design of experiments to SPARQL endpoint evaluation. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8219 LNCS, pp. 360–375). https://doi.org/10.1007/978-3-642-41338-4_23
Mendeley helps you to discover research relevant for your work.