Evaluation of techniques to detect wrong interaction based trace links

Abstract

[Context and Motivation] In projects where trace links are created and used continuously during the development, it is important to support developers with an automatic trace link creation approach with high precision. In our previous study we showed that our interaction based trace link creation approach achieves 100% precision and 80% relative recall and thus performs better than traditional IR based approaches. [Question/problem] In this study we wanted to confirm our previous results with a data set including a gold standard created by developers. Moreover we planned further optimization and fine tuning of our trace link creation approach. [Principal ideas/results] We performed the study within a student project. It turned out that in this study our approach achieved only 50% precision. This means that developers also worked on code not relevant for the requirement while interactions were recorded. In order to improve precision we evaluated different techniques to identify relevant trace link candidates such as focus on edit interactions or thresholds for frequency and duration of trace link candidates. We also evaluated different techniques to identify irrelevant code such as the developer who created the code or code which is not related to other code in an interaction log. [Contribution] Our results show that only some of the techniques led to a considerably improvement of precision. We could improve precision almost up to 70 % while keeping recall above 45% which is much better than IR-based link creation. The evaluations show that the full benefits of an interaction based approach highly depend on the discipline of the developers when recording interactions for a specific requirement. Further research is necessary how to support the application of our approach in a less disciplined context.