Scheduling high performance data mining tasks on a data grid environment

Abstract

Increasingly the datasets used for data mining are becoming huge and physically distributed. Since the distributed knowledge discovery process is bothdata and computational intensive, the Grid is a natural platform for deploying a high performance data mining service. The focus of this paper is on the core services of such a Grid infrastructure. In particular we concentrate our attention on the design and implementation of specialized broker aware of data source locations and resource needs of data mining tasks. Allocation and scheduling decisions are taken on the basis of performance cost metrics and models that exploit knowledge about previous executions, and use sampling to acquire estimate about execution behavior.