Using surveys and Web-scraping to select tools for software testing consultancy

Abstract

We analyzed findings from data collected utilizing surveys and Webscraping, to support Knowit Oy, a software testing consultation company, in the process of selecting the right tools for software testing & test automation. We conducted two surveys (2013 & 2016) among (mostly Finnish) software professionals to acquire criteria and a list of tools used for software testing in industry. Considering all our data sources Selenium was the most popular pure tool, while Robot Framework was the most referenced tool (latter survey). According to the surveys Jenkins and Sikuli have the highest increase in popularity (or familiarity). Top referred criteria for selection were usability, functionality, maintainability and available support for a tool. While Knowit considers it best to utilize traditional surveys, Web-scraping is seen as cost effective support for such instruments. To get comprehensive picture and to gain knowledge of the tools in markets multiple sources should be used.