Systems support for scalable data mining

Abstract

The current generation of data mining tools have limited capacity and performance, since these tools tend to be se- quential. This paper explores a migration path out of this bottleneck by considering an integrated hardware and soft- ware approach to parallelize data mining. Our analysis shows that parallel data mining solutions require the fol- lowing components: parallel data mining algorithms, paral- lel and distributed data bases, parallel file systems, parallel I/O, tertiary storage, management of online data, support for heterogeneous data representations, security, quality of service and pricing metrics. State of the art technology in these areas is surveyed with an eye towards an integration strategy leading to a complete solution.