A Case for Data-Driven Agile Transformations: Can Longitudinal Backlog Data Help Guide Organizational Improvement Journeys?

Abstract

Context: Almost every organization with a strong digital capability has embarked on an agile transformation journey. But do these changes actually deliver on the envisioned transformation goals? What conclusions can we draw from measurements and observations? Objective: The ambition of this report is to (1) assess whether tooling data can be used to guide a transformation towards improved organizational performance; (2) verify claimed benefits of agile transformations using tooling data in the presented case study. Method: We measure productivity, time-to-market, and quality as transformation objectives by analyzing longitudinal Jira backlog tooling data within an embedded multiple-unit case study. Results: By analyzing over 57,000 Jira issues from eight agile release trains over a period of three years, we (1) provide a proof of concept of how tooling data can be used to guide agile transformations; (2) provide empirical evidence on the assessment of transformation objectives over time and organizational layers at FinOrg; and (3) connect measurement results with available literature. Conclusions: We may conclude that tooling data is a viable addition to guide transformations through identification of improvement opportunities on the set objectives. We connected the case study results to existing literature and identified similarities. We argue that there is a need for a measurement framework and better understanding of the dynamics between measurement and performance.