Neural information processing in hierarchical prototypical networks

Abstract

Prototypical networks (PTNs), which classify unseen data points according to their distances to the prototypes of classes, are a promising model to solve the few-shot learning problem. Mimicking the characteristics of neural systems, the present study extends PTNs in two aspects. Firstly, we develop hierarchical prototypical networks (HPTNs), which construct prototypes at all layers and minimize the weighted classification errors of all layers. Applied to two benchmark datasets, we show that a HPTN has comparable, or slightly better, performances than a PTN. We also find that after training, the HPTN generates good prototype representations in the intermediate layers of the network. Secondly, we demonstrate that the classification operation via distance computation in a PTN can be replaced approximately by the attracting dynamics of the Hopfield model, indicating the potential realization of metric-learning in neural systems. We hope this study establishes a link between PTNs and neural information processing.