Correlation and Convolution

Abstract

Correlation and Convolution are basic operations that we will perform to extract information from images. They are in some sense the simplest operations that we can perform on an image, but they are extremely useful. Moreover, because they are simple, they can be analyzed and understood very well, and they are also easy to implement and can be computed very efficiently. Our main goal is to understand exactly what correlation and convolution do, and why they are useful. We will also touch on some of their interesting theoretical properties; though developing a full understanding of them would take more time than we have. These operations have two key features: they are shift-invariant, and they are linear. Shift-invariant means that we perform the same operation at every point in the image. Linear means that this operation is linear, that is, we replace every pixel with a linear combination of its neighbors. These two properties make these operations very simple; it’s simpler if we do the same thing everywhere, and linear operations are always the simplest ones. We will first consider the easiest versions of these operations, and then generalize. We’ll make things easier in a couple of ways. First, convolution and correlation are almost identical operations, but students seem to find convolution more confusing. So we will begin by only speaking of correlation, and then later describe convolution. Second, we will start out by discussing 1D images. We can think of a 1D image as just a single row of pixels. Sometimes things become much more complicated in 2D than 1D, but luckily, correlation and convolution do not change much with the dimension of the image, so understanding things in 1D will help a lot. Also, later we will find that in some cases it is enlightening to think of an image as a continuous function, but we will begin by considering an image as discrete, meaning as composed of a collection of pixels.