Deep learning does not replace Bayesian modeling: Comparing research use via citation counting

Abstract

One could be excused for assuming that deep learning had or will soon usurp all credible work in reasoning, artificial intelligence, and statistics, but like most “meme” class broad generalizations the concept does not hold up to scrutiny. Memes do not generally matter since the experts will always know better; but in the case of Bayesian software like Stan and PyMC3, even their developers and advocates bemoan the apparent dominance of deep learning as manifested in popular culture, breathtaking performance, and most problematically from funding agency peer review that impacts our ability to further advance the field. The facts, however, do not support the assumed dominance of deep learning in science upon closer examination. This letter simply makes the argument by the crudest of possible metrics, citation count, that once the discipline of Computer Science is subtracted, Bayesian software accounts for nearly a third of research citations. Stan and PyMC3 dominate some fields, PyTorch, Keras, and TensorFlow dominate others with lot of variations in between. Bayesian and deep-learning approaches are related but very different technologies in goals, implementation, and applicability with little actual overlap--so this is not a surprise. For example, deep learning cannot bring the explainability of applied math/statistics and Bayesian methods do not scale to deep-learning data sets. While deep-learning behemoths like Facebook and Google use and support Bayesian efforts, the Bayesian packages scientists actually use are academic/volunteer efforts punching far above their weight class, and they need financial support. It would behoove funders to fully understand the impact and role of Bayesian methods in resource allocation.