Divergence Measures as a Generalised Approach to Quantitative Emergence

Abstract

A technical system exhibits emergence when it has certain propertiesor qualities that can be termed to be irreducible in the sense thatthey are not traceable down to the constituent parts of the system.The article summarises three techniques for emergence detection andemergence measurement that were proposed by members of the OrganicComputing community. These techniques are based on information-theoreticand probabilistic viewpoints: the discrete entropy difference discussedin detail in the previous article, the Hellinger distance which isa divergence measure for probability densities, and an iterativeapproach motivated by divergence measures. Advantages and drawbacksof these measures are demonstrated by means of some simulation experimentsusing artificial data sets. It is shown that these techniques areable to deal with different kinds of emergent phenomena such as transitionsfrom chaos to order, concept drift, or novelty. That is, with thesetechniques it is possible to cover a wide range of possible applications.