Who is the Winner? Memristive-CMOS Hybrid Modules: CNN-LSTM Versus HTM

Abstract

Hierarchical, modular and sparse information processing are signature characteristics of biological neural networks. These aspects have been the backbone of several artificial neural network designs of the brain-like networks, including Hierarchical Temporal Memory (HTM). The main contribution of this work is showing that Convolutional Neural Network (CNN) in combination with Long short term memory (LSTM) can be a good alternative for implementing the hierarchy, modularity and sparsity of information processing. To demonstrate this, we draw a comparison of CNN-LSTM and HTM performance on a face recognition problem with a small training set. We also present the analog CMOS-memristor circuit blocks required to implement such a scheme. The presented memristive implementations of the CNN-LSTM architecture are easier to i mplement, train and offer higher recognition performance than the HTM. The study also includes memristor variability and failure analysis.