A requirements analysis for parallel KDD systems

Abstract

The current generation of data mining tools have limited capacit y and performance, since these tools tend to be sequential. This paper explores a migration path out of this bottlenec kby considering an in tegrated hardware and softw are approach to parallelize data mining. Our analysis shows that parallel data mining solutions require the following components: parallel data mining algorithms, parallel and distributed data bases, parallel file systems, parallel I/O, tertiary storage, management of online data, support for heterogeneous data representations, securit y, qualit yof service and pricing metrics. State of the art technology in these areas is surveyed with an eye tow ards an integration strategy leading to a complete solution. © 2000 Springer-Verlag Berlin Heidelberg.