Selection of DevOps best test practices: A hybrid approach using ISM and fuzzy TOPSIS analysis

Abstract

Testing is a complex phase in DevOps process due to need of an automated process that provides feedback at different strategies of continuous development and operations pipeline. Software organization face several challenges during the testing phase due to lack of understanding on testing best practices for the DevOps paradigm. The objective of this study is to prioritize DevOps best testing practices, which can facilitate the selection of testing practices during DevOps process. To perform this research, we have extended the work done by Hornbeek, using the 15 DevOps testing practices discussed in his study. First, we categorize the test practices against culture, automation, lean, measurement, and sharing (CALMS) pillars of DevOps adoption principles. Next, a questionnaire-based survey was conducted to collect feedback from industry practitioners on the DevOps test practices and their categorization against CALMS criteria. Finally, we applied Interpretive Structure Modeling (ISM) to find the interrelationship between CALMS criteria, and fuzzy TOPSIS was used to prioritize the DevOps test practices that will assist practitioners to better manage the testing activities during DevOps process.