Compressive classification: Where wireless communications meets machine learning

Abstract

This chapter introduces Shannon-inspired performance limits associated with the classification of low-dimensional subspaces embedded in a high-dimensional ambient space from compressive and noisy measurements. In particular, it introduces the diversity-discrimination tradeoff that describes the interplay between the number of classes that can be separated by a compressive classifier—measured via the discrimination gain—and the performance of such a classifier—measured via the diversity gain—and the relation of such an interplay to the underlying problem geometry, including the ambient space dimension, the subspaces dimension, and the number of compressive measurements. Such a fundamental limit on performance is derived from a syntactic equivalence between the compressive classification problem and certain wireless communications problems. This equivalence provides an opportunity to cross-pollinate ideas between the wireless information theory domain and the compressive classification domain. This chapter also demonstrates how theory aligns with practice in a concrete application: face recognition from a set of noisy compressive measurements.