The Definition
The integral
$$\int g(t-\tau )h(\tau) d\tau = g \otimes h$$
is called the convolution of \(g\) and \(h\). Here the \(\otimes\) denotes
convolution, it is sometimes represented by a \(\star\). This is representing the
amount of overlap of function \(h\) as \(g\) is shifted over it. It in effect
blends one function with another. This crops up a lot in physics and in
statistics.
The Fourier Transform of a Convolution
If \(F(u)\) is the Fourier transform of \(f(x)\) and \(G(u)\) is the Fourier
transform of \(g(x)\), multiplying both Fourier transforms together we have
$$F(u)G(u) = \int_{-\infty}^{\infty}f(x)e^{-2 \pi iu x} dx \int_{-\infty}^{\infty}g(x)e^{-2 \pi i u x} dx$$
So that we can rewrite this as a double integral we replace the \(x\) with
two dummy variables and rewrite as
$$F(u)G(u) = \int_{-\infty}^{\infty}f(\tau)e^{-2 \pi i u \tau} d \tau \int_{-\infty}^{\infty}g(\upsilon)e^{-2 \pi i u \upsilon} d
\upsilon$$
$$= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty}e^{-2 \pi i u(\tau + \upsilon)}f(\tau)g(\upsilon)d \tau d
\upsilon$$
Next, making a change of variables so \(x= \tau+\upsilon ,dx=d \upsilon\)
$$F(u)G(u) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(\tau) g(x-\tau) d \tau dx$$
$$=\int_{-\infty}^{\infty} e^{-i2 \pi x }[ \int_{-\infty}^{\infty}f(\tau) g(x-\tau) d\tau ] dx$$
$$=\int_{-\infty}^{\infty}e^{-i2 \pi x} f(x) \otimes g(x) dx$$
i.e., \(F(u)G(u)\) and \(f(x) \otimes g(x)\) are Fourier transform pairs. This
result is very important in optical processing, and signal processing in
general. It means a convolution can be calculated by Fourier transforming
both signals, multiplying the result and inverse transforming back. This is
often quicker than calculating the convolution directly and can be performed
using a simple system of lenses and filters.
Properties
Commutativity
$$f \otimes g = g \otimes f$$
Associativity
$$f \otimes (g \otimes h) = (f \otimes g) \otimes h$$
Distributivity
$$f \otimes (g+h)= (f \otimes g) + (f \otimes h)$$
Scalar multiplication
$$a(f \otimes g) = (af) \otimes g = f \otimes (ag)$$
where \(a \in \mathbb{C}\)
Differential rule
If \(\partial\) is a differential operator
$$\partial (f \otimes g) = \partial f \otimes g = f \otimes \partial g$$
A graphical example of convolution/correlation
