Residual networks (ResNet) and their importance in machine vision

The computer vision It is a branch of artificial intelligence that aims to teach machines to «see» and interpret images and videos, emulating human visual ability.

This discipline has applications in a wide range of fields, such as medicine, security, the automotive industry and e-commerce. However, processing and analysing images efficiently requires complex models that can capture subtle details and intricate patterns.

With this in mind, the deep neural networks are fundamental to changing the way images are processed and understood for tasks such as image classification, facial recognition and object detection. However, as neural networks become deeper, technical challenges arise, such as difficulties in training the model and a loss of accuracy.

This is where it comes in ResNet, an architecture that brought about a paradigm shift in computer vision, enabling the training of much deeper networks without sacrificing efficiency or accuracy, and marking a turning point in deep learning.

ResNet: Transforming Computer Vision

The introduction of ResNet (Residual Networks) marked a turning point in the field of computer vision, overcoming the limitations that had existed up to that point thanks to the innovative incorporation of jump connections.

Designed in 2015 by Kaiming He and his team, ResNet made it possible to build deeper and more effective neural networks. This breakthrough not only improved performance in image classification tasks, but also had an impact on advanced applications such as object recognition and semantic segmentation.

The ResNet architecture introduces the concept of residual blocks, a key innovation for overcoming the problems associated with deep neural networks.

Rather than simply stacking layers, as is the case in traditional architectures, ResNet uses skip connections which allow the input of a layer to be added directly to its output after passing through two or more layers.

A residual block It can be represented mathematically as:

y = F(x, {Wi}) + x

Where:

• x This is the entrance to the block.
• F(x,{Wi}) represents the learnt non-linear transformation (e.g. convolutions, batch normalisation and ReLU).
• y It is the exit from the block.

This structure enables the network to learn residual adjustments rather than complete transformations, facilitating gradient propagation during training.

ResNet architecture

ResNet is available in various variants, such as ResNet-18, ResNet-34, ResNet-50 and ResNet-152, where the number indicates the total number of layers in each network. These variants differ mainly in the number and arrangement of residual blocks within the architecture:

• ResNet-18: contains 2 residual blocks on each of the 4 main levels, making a total of 8 residual blocks.
• ResNet-34: extends this structure to 3 residential blocks on two levels and 4 blocks on the other two, making a total of 16 residential blocks.
• ResNet-50: introduces residual blocks with deeper layers, known as ‘bottleneck’ residual blocks, organised as 3, 4, 6 and 3 blocks at the respective levels, with a total of 50 layers.
• ResNet-152: It significantly increases the depth with 3, 8, 36 and 3 residual blocks at each level, reaching a total of 152 layers.

These residual blocks are interspersed with convolution and pooling operations, allowing information to flow efficiently through the network, even in the deepest architectures, where the number indicates the total number of layers in the network. As the depth increases, the use of residual blocks and skip connections ensures that the networks are easier to train, even with hundreds of layers.

Some notable features of its architecture are:

1. Initial convolutions: a convolutional layer with a 7×7 kernel and a stride of 2, followed by batch normalisation and ReLU. This large kernel size allows general spatial features to be captured from the outset, whilst the stride reduces the image resolution, thereby lowering the computational load on the subsequent layers.
2. Residual blocks: designed to maintain efficient gradient propagation. Each block is organised into convolution layers that apply 3×3 filters or, in deeper variants such as ResNet-50 and above, a bottleneck-style design that uses 1×1 convolutions to reduce dimensionality before applying 3×3 filters, and then restores it with another 1×1 convolution. These configurations ensure a balance between computational efficiency and learning capacity.
3. Global pooling: reduces the dimensionality of the output prior to final classification. This reduction is achieved by applying an average (or, in some cases, a maximum operation) across the entire spatial dimension of the features extracted by the preceding layers. This not only reduces the number of parameters in the final layer, but also acts as an effective regularisation, improving the model’s generalisation. Furthermore, it makes the network more computationally efficient, as it minimises the amount of data processed in the final stages.
4. Sorter: the output of the final residual block passes through a global pooling stage, followed by a fully connected layer that reduces the features to the number of classes in the problem. Finally, a softmax function is applied to generate normalised probabilities, enabling image classification. This design ensures a smooth transition from the learnt features to the final prediction.

The revolution in deep neural networks with skip connections

Before ResNet, deep architectures such as VGG were finding it difficult to climb because of the vanishing gradient problem (gradient vanishing problem). The skip connections introduced by ResNet address this problem by creating alternative routes to facilitate the flow of information, reducing reliance on intermediate layers.

This design has proved particularly useful in “for image” tasks”, such as:

• Image classification: significantly improving accuracy rates on datasets such as ImageNet.
• Object recognition: using ResNet as the basis for architectures such as Faster R-CNN.
• Semantic segmentation: used in medical and autonomous driving contexts.

The use of jump connections has a number of advantages:

• Lossless propagation: they help information to flow down to the deeper layers.
• Improving training: they reduce the likelihood of saturation in deep networks.
• Computational efficiency: they allow the model to be trained using fewer resources.

How does ResNet improve accuracy in deep neural networks?

Essentially, ResNet is a residual network designed to tackle the critical problems associated with traditional deep neural networks. The key to its success lies in its ability to:

1. Achieving greater depth: Networks such as ResNet-152 have achieved new milestones, such as an accuracy of over 93% on ImageNet.
2. Simplifying optimisation: by learning residual adjustments rather than complete functions, the computational effort is reduced.
3. Generalise more effectively: its structure enables robust models to be obtained for a variety of applications.

Since its introduction, ResNet has been widely adopted in fields such as healthcare, commerce and scientific research. For example:

• In medical diagnosis, it enables the automatic detection of pathologies in X-ray images.
• In e-commerce, he optimises image-based recommendation systems.

Furthermore, its structure has inspired new architectural styles such as DenseNet, which expands on the concept of hop connections by linking all the layers together.

Revolution with residual blocks and jump connections

ResNet has revolutionised deep neural networks by introducing residual blocks and skip connections, thereby overcoming key limitations in the training of deep models. Their impact extends beyond computer vision, laying the foundations for innovation in various areas of artificial intelligence.

Thanks to its ability to train deep learning models and its versatility, ResNet remains a key benchmark in the design of modern neural network architectures.

If you want to learn how to programme neural networks such as ResNet and other architectures, I recommend our Neural Networks Course using Python.