On the need of opening the big data landscape to everyone: Challenges and new trends

Abstract

The great variety and intrinsic complexity of current Big Data technologies hampers the development of analytic processes for large data sets in domains where their business experts are not required to have specialized knowledge in computing, such as data mining, parallel computing, machine learning or software development. New approaches are therefore necessary to simplify, promote and open to everyone the establishment of these technologies in those sectors like health, economy, market analysis, etc., where such a data processing is highly demanded but it still needs to be outsourced. In this context, workflows are conceptually closer to the business expert, and a well-known mechanism to represent a sequence of domain-specific activities that enable the automation of data processes, independently of the infrastructure requirements. In this chapter, we discuss the current challenges to be faced in the widespread adoption of workflow-based Big Data processes. Further, existing workflow management tools are analyzed, as well as the new trends for the development of custom solutions in multiple domains.