Extracting Queryable Knowledge Graphs from User Stories: An Empirical Evaluation

Abstract

User stories are brief descriptions of a system feature told from a user's point of view. During requirements elicitation, users and analysts co-specify these stories using natural language. A number of approaches have tried to use Natural Language Processing (NLP) techniques to extract different artefacts, such as domain models and conceptual models, and reason about software requirements, including user stories. However, large collections of user story models can be hard to navigate once specified. We extracted different components of user story data, including actors, entities and processes, using NLP techniques and modelled them with graphs. This allows us to organise and link the structures and information in user stories for better analysis by different stakeholders. Our NLP-based automated approach further allows the stakeholders to query the model to view the parts of multiple user stories of interest. This facilitates project development discussions between technical team members, domain experts and users. We evaluated our tool on user story datasets and through a user study. The evaluation of our approach shows an overall precision above 96% and a recall of 100%. The user study with eight participants showed that our querying approach is beneficial in practical contexts.