Unit Testing for Concepts in Neural Networks

Abstract

Many complex problems are naturally un-derstood in terms of symbolic concepts. For example, our concept of ‘‘cat’’ is related to our concepts of ‘‘ears’’ and ‘‘whiskers’’ in a non-arbitrary way. Fodor (1998) proposes one theory of concepts, which emphasizes symbolic representations related via constituency structures. Whether neural networks are consistent with such a theory is open for debate. We propose unit tests for evaluating whether a system’s behavior is consistent with several key aspects of Fodor’s criteria. Using a simple visual concept learning task, we evaluate several modern neural architectures against this specification. We find that models suc-ceed on tests of groundedness, modularity, and reusability of concepts, but that important questions about causality remain open. Re-solving these will require new methods for analyzing models’ internal states.