Output Coding and Modularity for Multi-Class Problems

Abstract

We study backpropagation networks learning classification problemswith multiple classes k>3. The common way to code the output ofa network is the one-per-class (OPC) method, where one bit is assignedto each class. A technique called error-correcting output coding(ECOC) converts the k-class learning problem into a large numberof two-class learning problems. We propose to use modular architecturesas a way to decorrelate the (redundant) network outputs. Variousmodular architectures are tested on an artificial problem. We concludethat ECOC only improves upon OPC when combined with a sufficientlymodular approach.