Why some families of probability distributions are practically efficient: A symmetry-based explanation

Abstract

Out of many possible families of probability distributions, some families turned out to be most efficient in practical situations. Why these particular families and not others? To explain this empirical success, we formulate the general problem of selecting a distribution with the largest possible utility under appropriate constraints. We then show that if we select the utility functional and the constraints which are invariant under natural symmetries—shift and scaling corresponding to changing the starting point and the measuring unit for describing the corresponding quantity x— then the resulting optimal families of probability distributions indeed include most of the empirically successful families. Thus, we get a symmetry-based explanation for their empirical success.