The Value of Crowdsourcing for Complex Problems: Comparative Evidence from Software Developed By the Crowd And Professionals

Abstract

Crowdsourcing is " the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call " [10, 27, 28, 53]. Crowdsourcing has also been described as a problem-solving model [3, 9, 14, 15]; gaining input from many unknown and unconnected contributors [25]; and distributed production models that ask for contributions via open calls from an undefined large network of people [3, 54]. The common attribute of crowdsourcing in all these definitions is that it is a collaborative effort enabled by people-centric technology. Crowdsourcing business models benefit organizations by providing cheap labor and by tapping geographically disperse crowds. Critics of the wisdom of crowds suggest that collective wisdom may be only useful for simple problems, and may be difficult to use for complex problems such as software development. As the practice of problem solving with crowdsourcing becomes increasingly common, it is essential to identify whether the wisdom of crowds can be applied to solve complex problems. There are two alternative streams of research that focus on the legitimacy of the crowd's/customer's complex problem-solving abilities. One stream suggests that crowds are mostly novice and do not have sufficient domain expertise to participate in and solve complex problems such as product innovation and development [33, 43, 5]. The other stream argues that " innovation is being democratized " [51], meaning that crowds/customers of product and services know about their requirements, are able to contribute toward the development of a product, and can solve complex problems [9,32, 51]. Research on Complex Problem Solving (CPS) has revealed a wide variety of thoughts and insights about the characteristics and operationalization of complex problems [17]. The research community is still debating which definition should be widely accepted by the scientific community, what is " complex " in CPS, and how to evaluate the complexity of problems [42]. In group environments, CPS addresses challenges such as coordination of tasks, lack of domain expertise by community members, lack Abstract Crowdsourcing is a problem solving model. In the context of complex problems, conventional theory suggests that solving complex problems is a province of professionals, that is, people with sufficient knowledge about the domain. Prior literature has indicated that the crowd, in addition to professionals, is also a great source for solving problems such as product innovation and idea generation. However, this assumption has yet to be tested. Adopting a quasi-experimental approach, this study uses a two-phase process to investigate this question. In the first phase we compare the development of a software by the crowd and professionals. In the second phase we evaluate the software developed by the crowdsourcing business model and professionals in terms of key perceived quality dimensions assessed by users of the systems. Quality is measured in terms of pragmatic quality, hedonic quality stimulation, and hedonic quality identification. Our study results suggest that there is a statistically significant difference between the software developed by a crowdsourcing business model and professionals in terms of hedonic quality stimulation and hedonic quality identification but there is no difference in terms of pragmatic quality. This research offers a first assessment of whether a crowdsourcing business model can be used to develop software with better user experience than professionally-developed software.