Polystore data management systems for managing scientific data-sets in big data archives

Abstract

Large scale scientific data sets are often analyzed for the purpose of supporting workflow and querying. User need to query over different data sources. These systems manage intermediate results. Most prototypes are complex and have an ad hoc design. These require extensive modifications in case of growth of data and change of scale, in terms of data or number of users. New data sources may arise to further complicate the ad hoc design. The polystore data management approach provides ‘data independence’ for changes in data profile, including addition of cloud data resources. The users are often provided a quasi-relational query language. In many cases, the polystore systems support distinct tasks that are user defined workflow activity, in addition to providing a common view of data resources.