A novel agglomerative hierarchical approach for clustering in medical databases

Abstract

Agglomerative hierarchical clustering plays vital roles in large number of application areas such as medical, bioinformatics, information retrieval etc. It generates clusters by iteratively merging sub clusters and hence merging criterion is very critical for its performance. But most of existing techniques do not consider global features of clusters to decide upon which clusters should be merged and are even unable to undo a wrong merge once it is done. These factors contribute to their poor performance. Also existing techniques face challenges on real world databases because these are proposed either for numeric or categorical data but real world data contains mixed attributes for example medical databases. To address the above mentioned drawbacks, in this paper, we propose a novel agglomerative hierarchical clustering method which avails the spread of data clusters as merging criterion. It is helpful in considering overall distribution of clusters in merging them. Variance and entropy are employed to measure the spread of a cluster in numeric and categorical attributes respectively. To counter the effect of a wrong merge, proposed method allows reallocation of data objects between clusters. We have experimented on real life medical databases and results show the efficacy of proposed approach.