Probing knowledge in distributed data mining

Abstract

In this paper, we propose a new approach to apply metalearning concept to distributed data mining. We name this approach Knowledge Probing where a supervised learning process is organised into two learning stages. In the first learning phase, a set of base classifiers are learned in parallel from a distributed data set. In the second learning phase, meta-learning is applied to induce the relationship between an attribute vector and the class predictions from all the base classifiers. By applying this approach to an environment where base classifiers are produced from distributed data sources, the output of Knowledge Probing process can be viewed as the assimilated knowledge of that distributed learning system. Some initial experimental results on the quality of the assimilated knowledge are presented. We believe that an integration of Knowledge Probing technique and the available data mining algorithms can provide a practical framework for distributed data mining applications.