Mutual information estimation in higher dimensions: A speed-up of a k-nearest neighbor based estimator

Abstract

We focus on the recently introduced nearest neighbor based entropy estimator from Kraskov, Stögbauer and Grassberger (KSG) [10], the nearest neighbor search of which is performed by the so called box assisted algorithm [7]. We compare the performance of KSG with respect to three spatial indexing methods: box-assisted, k-D trie and projection method, on a problem of mutual information estimation of a variety of pdfs and dimensionalities. We conclude that the k-D trie method is significantly faster then box-assisted search in fixed-mass and fixed-radius neighborhood searches in higher dimensions. The projection method is much slower than both alternatives and not recommended for practical use. © Springer-Verlag Berlin Heidelberg 2007.