DETR IMAGE INFERENCE MODEL

A NAAN MUDHALVAN REPORT

Submitted by

SURYA S S
(711721104117)

in partial fulfillment of the award of the degree

Of

BACHELOR OF ENGINEERING

IN

DEPARTMENT OF COMPUTER SCIENCE AND

ENGINEERING

KGiSL INSTITUTE OF TECHNOLOGY

ANNA UNIVERSITY: CHENNAI 600 025

NOVEMBER 2023
 BONAFIDE CERTIFICATE

Certified that this Naan Mudhalvan report “DETR Image Inference Model” is the
bonafide work of “Surya S S” who belongs to III Year Computer Science and
Engineering “B” during the Sixth Semester of Academic Year 2023-2024.

FACULTY IN CHARGE HEAD OF THE DEPARTMENT

Certified that the candidates were examined by us for Naan Mudhalvan Practical
Viva held on …………………. at KGiSL Institute of Technology, Saravanampatti,
Coimbatore 641035.

INTERNAL EXAMINER EXTERNAL EXAMINER

 ACKNOWLEDGEMENT

We express our deepest gratitude to our Chairman and Managing Director Dr.
Ashok Bakthavachalam for providing us with an environment to complete our
Internship project successfully.

We are grateful to our CEO of Academic Initiatives Mr Aravind Kumar

Rajendran and our beloved Secretary Dr Rajkumar N Our sincere thanks to
honourable Principal Dr Suresh Kumar S and Academic Director Dr Shankar
P for his support, guidance, and blessings.

We would like to thank Dr Thenmozhi S, Head of the Department, and Naan

Mudhalvan Coordinator Ms Lathika B A, Department of Computer Science and
Engineering for her firm support during the entire course of this Internship and
who modeled us both technically and morally for achieving greater success in this
project work.

We also thank all the faculty members of our department for their help in making
this Internship project successful. Finally, we take this opportunity to extend our
deep appreciation to our Family and Friends, for all they meant to us during the
crucial times of the completion of our project
 TABLE OF CONTENTS

● Abstract
● Introduction
● Background
○ DETR Architecture
○ Object Detection Methods
○ Importance of Efficient Object Detection
● Methodology
● Implementation
○ Environment Setup
○ Loading Pre-trained Model
○ Image Encoding Process
○ Inference Procedure
○ Post-processing Techniques
○ Visualization Methods
● Results
ABSTRACT

This project implements an image inference model based on Facebook's DETR

(DEtection TRansformer) architecture for efficient object detection. Unlike
traditional methods, DETR directly predicts bounding boxes and class labels in a
single pass, utilizing transformer-based neural networks. The report details the
methodology, including initialization, image encoding, inference, post-processing,
and visualization. Implementation specifics cover environment setup, loading pre-
trained models, and applying post-processing techniques. Results showcase the
model's performance metrics and visualized detection results. Discussions
highlight DETR's advantages, comparisons with traditional methods, and avenues
for future research. Overall, this project contributes to advancing object detection
methods by leveraging transformer architectures and provides insights for further
development in the field.
INTRODUCTION

Object detection is a fundamental task in computer vision, with numerous

applications ranging from autonomous driving to surveillance systems. Traditional
object detection methods often rely on complex architectures and handcrafted
features, leading to inefficiencies and limitations. In contrast, Facebook's DETR
(DEtection TRansformer) architecture offers a novel approach by leveraging
transformer-based neural networks for object detection.

The primary objective of this project is to implement an image inference model

based on the DETR architecture. By utilizing pre-trained weights and efficient
techniques, the model aims to provide accurate and fast object detection
capabilities.

This project focuses on the implementation and evaluation of the DETR-based

image inference model. It covers the entire pipeline from initialization to
visualization of detection results. The scope also includes discussions on the
methodology, implementation details, and analysis of results.
BACKGROUND

DETR Architecture
DETR is a transformer-based neural network designed specifically for object
detection tasks. Unlike traditional methods that rely on anchor boxes and complex
pipelines, DETR directly predicts bounding boxes and class labels in a single pass.
This architecture consists of encoder and decoder layers, allowing it to capture
spatial information and relationships between objects effectively.

Object Detection Methods

Traditional object detection methods include techniques like region-based
convolutional neural networks (R-CNN), You Only Look Once (YOLO), and
Single Shot MultiBox Detector (SSD). While these methods have been successful,
they often suffer from limitations such as slow inference speeds and complex
training procedures.

Importance of Efficient Object Detection

Efficient object detection is crucial for real-time applications where speed and
accuracy are essential. By adopting transformer-based architectures like DETR, it
becomes possible to achieve both efficiency and effectiveness in object detection
tasks.
METHODOLOGY

In the initialization phase, the pre-trained DETR model is loaded along with the
feature extractor, ensuring that all necessary components are prepared for
subsequent stages. This process involves not only loading the model weights but
also configuring the model architecture and associated parameters to ensure
compatibility with the input data and desired inference tasks.

During the image encoding step, the feature extractor analyzes the input image,
extracting salient features that are crucial for accurate object detection. This
process typically involves multiple layers of convolutional operations, where
features are progressively abstracted to capture hierarchical representations of the
input image. The encoded representation obtained from this step serves as a rich
feature map that encapsulates the semantic information necessary for effective
object localization and classification.

Upon encoding, the encoded image data is forwarded to the DETR model for
inference. Here, the model utilizes its transformer-based architecture to process the
encoded features and generate predictions regarding the presence, location, and
class labels of objects within the image. The transformer architecture enables the
model to capture long-range dependencies and spatial relationships between
objects, contributing to its superior performance in object detection tasks.

Following inference, post-processing techniques are employed to refine the

predicted bounding boxes and eliminate spurious detections. Techniques such as
non-maximum suppression (NMS) are applied to suppress redundant bounding
boxes, retaining only the most confident predictions for each object instance.
Additionally, a confidence threshold may be imposed to filter out detections with
low confidence scores, ensuring that only reliable detections are considered in the
final results.

Finally, the detection results are visualized by overlaying bounding boxes and class
labels onto the input image. This visualization step provides a comprehensive view
of the object detection outcomes, allowing users to easily interpret and assess the
performance of the model. By visually annotating the image with detection results,
the effectiveness of the model in accurately localizing and classifying objects can
be readily observed, facilitating further analysis and decision-making.
IMPLEMENTATION

Environment Setup
In the environment setup phase, the project dependencies and libraries are installed
and configured to create a conducive development environment. This typically
involves using package managers like pip or conda to install essential libraries
such as PyTorch and torchvision. These libraries provide the foundational tools
and frameworks necessary for implementing the object detection model based on
the DETR architecture. Additionally, any specific hardware requirements or GPU
accelerators are configured to leverage hardware acceleration for improved
performance during model training and inference.

Loading Pre-trained Model

After setting up the environment, the pre-trained weights of the DETR model
trained on the COCO dataset are loaded. These pre-trained weights contain the
learned parameters of the model, enabling it to make accurate predictions on new
input data. By loading pre-trained weights, the model benefits from transfer
learning, where knowledge gained from training on a large dataset is transferred to
a new task or domain. This facilitates quick deployment of the object detection
model, as the model is already initialized with learned representations of objects
and their features.

Image Encoding Process

Once the pre-trained model is loaded, the input image undergoes the image
encoding process. This involves passing the image through the feature extractor,
which is typically a convolutional neural network (CNN) architecture such as
ResNet or VGG. The feature extractor analyzes the input image, extracting
hierarchical representations of features at different spatial scales. These features
capture essential visual information such as edges, textures, and object shapes,
which are crucial for accurate object detection. The output of the feature extractor
is an encoded representation of the input image, often referred to as a feature map,
which is then passed to the DETR model for inference.

Inference Procedure
In the inference procedure, the encoded image obtained from the feature extractor
is fed into the DETR model. The DETR model processes the encoded features
using its transformer-based architecture to generate predictions of bounding boxes
and class labels for detected objects within the image. The model utilizes self-
attention mechanisms to capture long-range dependencies and spatial relationships
between objects, enabling it to make accurate and contextually informed
predictions. The output of the inference procedure is a set of bounding boxes along
with their corresponding class labels, representing the detected objects within the
input image.

Post-processing Techniques
Following inference, various post-processing techniques are applied to refine the
detection results and improve their accuracy. One common post-processing
technique is non-maximum suppression (NMS), which suppresses redundant
bounding boxes by selecting only the most confident detections and eliminating
overlapping boxes. Additionally, a confidence threshold may be applied to filter
out detections with low confidence scores, ensuring that only reliable detections
are retained. These post-processing techniques help improve the precision and
reliability of the detection results, making them suitable for further analysis or
application.

Visualization Methods
Finally, the detected objects are visualized using bounding boxes overlaid onto the
input image. Each bounding box is drawn around a detected object, with its
corresponding class label displayed alongside. This visualization provides a clear
and intuitive representation of the object detection outcomes, allowing users to
easily interpret and assess the performance of the model. By visualizing the
detection results, any inaccuracies or misclassifications can be identified and
addressed, contributing to the refinement and improvement of the object detection
model.
RESULTS

The detection results obtained from the object detection model are visualized
through images annotated with bounding boxes and class labels. Each bounding
box encapsulates a detected object, with its corresponding class label displayed
alongside. The visualization provides a tangible representation of the model's
capabilities, allowing users to easily interpret and assess the accuracy of the
detections. By visually inspecting the annotated images, any inaccuracies or
misclassifications can be identified, facilitating further refinement and
improvement of the model. Additionally, the visualized detection results serve as a
means of validation, enabling stakeholders to verify the correctness of the
detections and assess the model's suitability for specific applications. Overall, the
visualized detection results showcase the effectiveness of the object detection
model in accurately localizing and classifying objects within images,
demonstrating its practical utility in real-world scenarios.