Open in app Sign up Sign in

Search

Convolution Neural Networks: All you need to Know

Rukaiya Bano · Follow
4 min read · May 30, 2023

Listen Share

Convolution neural networks are a type of neural network that are generally used for image/video processing and
analyzing time series data (data that has a known grid-like topology). The convolution networks perform
convolutions on a segment of an image instead of traditional matrix multiplication.

The convolution neural networks are suitable for image processing as they take into account the neighboring
pixels, as for images the neighboring pixels of a pixel get a say in defining it. Convolution neural networks
leverage three important ideas that can help improve machine learning systems: sparse interactions, parameter
sharing, and equivariant representations. Moreover, convolution neural networks provide a means for working
with inputs of variable size.

Sparse Connectivity
The traditional neural networks use matrix multiplication by a matrix of parameters with a separate parameter
describing the interaction between each input and output unit. This means that every input unit is connected to
every output unit. Convolution neural networks, however, have sparse connections (also known as sparse
weights).
 Traditional Neural Networks

Convolution Neural Networks

This is accomplished by making the kernel( a matrix of weights that are multiplied with the input to extract
relevant features)smaller than the input image. The kernels can detect small, meaningful features taking up only a
small part of the image. This means that we only need to store a few parameters. This allows the network to
efficiently describe complicated interactions between many variables by constructing interactions from simple
building blocks which each describe only sparse interactions.

Parameter Sharing
Parameter sharing refers to the use of the same parameters for more than one function in a model. In a traditional
neural net, each element of the neural network has its own weight, i.e. each element of the weight matrix is used
only once while computing the output of a layer. It is multiplied by one element of the input and then never
revisited.
We can also say that the convolution networks have tied weights because the weights applied to one input are tied
to the value of the weight applied elsewhere. This means that rather than learning a different set of parameters for
each location, we use only one set.

Equivariance Representation
In convolutions, the particular case of parameter sharing causes the networks to have a property called
equivariance. To say a function is equivariant means that if the input changes, the output changes in the same way.

When processing time series data, this means that convolution produces a sort of timeline of when different
features appear in the input. If we move an event later in time, the exact same representation of it will appear in
the output, just later in time.

Similarly, for images, convolution creates a 2-D map of where certain features appear in the input. If we move the
object in the input, it will move by the same amount in the output. This is useful when we know that some
function of the same number of pixels is useful when applied to multiple input locations.

How Convolution Neural Networks are time and space efficient?

Let m be the number of inputs and n be the number of outputs. For traditional networks, we would require m*n
parameters and O(m*n) runtime. When we limit the number of connections to k using convolution networks the
total parameters required will be k*n and the runtime will be O(k*n) where k<m. The k can be several dimensions
less than m.

Also, since we use the idea of parameter sharing, we only need to store k parameters, thus reducing the memory
requirement. Convolution is thus dramatically more efficient than matrix multiplication in terms of memory
requirements and statistical efficiency.

Convolutional Neural Networks mark a significant advancement in the field of machine learning and artificial
intelligence, particularly in image and video processing. They are not only efficient in terms of computational
resources but also proficient at capturing the hierarchical pattern in data. By implementing sparse connectivity,
parameter sharing, and equivariance representation, CNNs are able to handle complex visual tasks that are
usually challenging for traditional neural networks.

Whether it’s facial recognition in security systems or diagnosing diseases from medical imagery, the applications
of CNNs are immense and continue to expand. As we continue to generate more and more data, it is undeniable
that the role of CNNs will become even more crucial. If we harness their power efficiently and ethically, we’re just
scratching the surface of what’s possible.

Artificial Intelligence Convolutional Network Neural Networks Deep Learning Computer Vision