Dynamic Thresholds for Self-Organizing Predictive Cells

Abstract

It has become increasingly popular to view the brain as a prediction machine. This view has informed a number of theories of brain function, the most prominent being predictive processing, where generative hypotheses are iteratively updated by error signals. In this treatment we take a lower level approach by examining the hierarchical temporal memory framework, which views individual pyramidal cells as the primary predictive unit of a self-organizing networked sequence learning system. Within this computational framework, the cell behaviour is constrained by a number of parameters which are static and shared across all cells. To further increase the adaptability of the cells, we shift away from this paradigm by introducing the concept of dynamic thresholds. This allows for the activation threshold (the amount of activity on a distal dendrite needed to form a prediction) to be adjusted continuously and individually for each cell. As a metric we use the prior, or unconditional, probability of activity on the proximal dendrites. Our experiments show that using this metric for dynamic thresholds can improve the predictive capabilities of the system in a number of domains, including anomaly detection, where we achieve state-of-the-results on the Numenta Anomaly Benchmark.