Object Detection withTensorflow

o D. HARI VAMSHI
o V. RAJU
o U. LAXMAN
 Agenda

➢ Intro
➢ What is Object Detection
➢ State of Object Detection
➢ Tensorflow Object DetectionAPI
➢ Preparing Data
➢ Training & Evaluating Models
➢ Links
What is Object Detection

Object detection is the

task of
identifying objects in an
image and drawing
bounding boxes around
them, i.e. localizing them.
It's a very important
problem in computer
vision due its numerous
applications from self-
driving cars to security and
tracking.
 Object detection =
Object Classification + ObjectLocalization
 Approaches

▪ Classicalapproach(Haarfeatures)-firstOD realtimeframework(Viola-Jones)
▪ Deeplearningapproach-now stateoftheart inOD
▪ OverFeat
▪ R-CNN
▪ FastR-CNN
▪ YOLO
▪ FasterR-CNN
▪ SSDandR-FCN
 Deep learning approach

OverFeat -publishedin 2013,multi-scalesliding

window algorithm using Convolutional Neural
Networks (CNNs).

N.N - RegionswithCNN features.Threestage

approach:
- Extractpossibleobjectsusingaregionproposal
method(themostpopularonebeingSelective
Search).
- ExtractfeaturesfromeachregionusingaCNN.
- ClassifyeachregionwithSVMs.
 Deep learning approach
Fast R-CNN - Similar to R-CNN, it used
Selective Search to generateobjectproposals,but
instead of extractingallofthemindependentlyand
usingSVM classifiers,itappliestheCNN onthe
completeimage and then used both Regionof
Interest (RoI)Pooling onthefeaturemapwitha
finalfeedforwardnetwork for classificationand
regression.

YOLO - You Only LookOnce:a

simple convolutional neural
network approach which has
both great results and high
speed,allowingfor thefirsttime
real timeobjectdetection.
 Deep learning approach

Faster R-CNN - Faster R-CNN added what

theycalledaRegionProposalNetwork(RPN),
in anattemptto getrid oftheSelectiveSearch
algorithm and make the model completely
trainableend-to-end.

SSD andR-FCN
Finally, there are two notable papers, Single Shot
Detector (SSD)whichtakesonYOLObyusingmultiple
sized convolutional feature maps achieving better
results and speed, and Region-based Fully
Convolutional Networks (R-FCN) which takes the
architecture of Faster R-CNN but with only
convolutionalnetworks.
Introduction

TensorFlow is a free and open-

source software library for
dataflow and differentiable
programming across a range of
tasks. It is a symbolic math
library, and is also used for
machine learning applications
such as neural networks
Tensorflow Object
Detection API
Creating a dataset

✓ We can either create a dataset of

our own or we can also consider
a predefined dataset and work on
the basis of TensorFlow package.

✓ The dataset being considered in

this project is CIFAR-10 which
consists of numerous pictures
used in detection classified in 10
classes.
 Dataset

➢ Tensorflow Object Detection API uses the TF

Record file format
➢ There is available third-
party scripts to convert PASCAL VOC and
Oxford Pet Format
➢ In other cases explanation of format
is available in git repo.
➢ Input data to create TFRecord - annotated
image
 Getting images

Grab from internet Create own images

➢ Scrap images from googleor Pixa
bay orwhatever
➢ For batch downloading -
Faktun Bulk Image Downlo
ader
➢ For data mining by multiplying existing
images - ImageMagic
➢ Record video with needed object/object
s (in640x480)
➢ Process video and split on screenshots.
➢ Create images with different
lights, background and so on.
➢ If object is able to have different form
s -better to catch themal .
➢ Try to make30%-
50%of images with overlaid object
➢ Tool for image augmentation
 Labeling ( extensions to be given to the files)

INPUT : The images can be of any

extension as, either .JPG, .PNG and soon.

OUTPUT : .xml files with bounding

boxes coordinates
➢ WHEN OUTPUT DATA HAS A DIFFERENT STRUCTURE:
• PREPARING INPUTS:
To use your own dataset in Tensorflow Object Detection API, you must convert it into
the TFRecord file format.

• LABEL MAPS:
Each dataset is required to have a label map associated with it. This label map defines a mapping
from string class names to integer class Ids. The label map should be a StringIntLabelMap text
protobuf and can be found in object detection/data.

• DATASET REQUIREMENTS:
For every example in your dataset, you should have the following information:
o An RGB image for the dataset encoded as jpeg or png.
o A list of bounding boxes for the image. Each bounding box should contain:
o A bounding box coordinates (with origin in top left corner) defined by 4 floating point
numbers [ymin, xmin, ymax, xmax]. Note that we store the normalized coordinates (x /
width, y / height) in the TFRecord dataset.
o The class of the object in the bounding box.
 Creating TFRecord

➢ Tensorflow object detection API report contains folder dataset_tools with script
s to covert common structures of data into TFRecord.

➢ If output data has another structure then , it includes to follow few steps
described in next slide.
 One model for twotasks?

Po - is objectexists

bx1
bx2 - bounding box
by1 coordinates
Objectdetection-outputistheonenumber(index)ofaclass by2

c1
c2
c3 - object’s
… variables
cn

Objectlocalization-outputisthefournumbers-
coordinatesofboundingbox.
Training
 Selecting a model

Tensorflow OD API provides a

collection of detection models pre-
trained on the COCO dataset, the Kitti dataset,
and the Open Images dataset.

- model name corresponds to a config file that

was used to train this model.
- speed-
running time in msper 600x600 image
- mAP stands for mean average precision,
which indicates how well the model
performed on the COCO dataset.
- Outputs types (Boxes, and Masks if ap
plicable)
 Training & Evaluating

# From the tensorflow/models/research directory

python object_detection/train.py
--logtostderr
--
pipeline_config_path=/tensorflow/models/object_detection/samples/configs/ssd_mobilenet_v1_p
ets.config
--train_dir=${PATH_TO_ROOT_TRAIN_FOLDER}

# From the tensorflow/models/research directory

python object_detection/eval.py \
--logtostderr \
--pipeline_config_path=${PATH_TO_YOUR_PIPELINE_CONFIG} \
--checkpoint_dir=${PATH_TO_TRAIN_DIR} \
--eval_dir=${PATH_TO_EVAL_DIR}
 Facial Recognition:
A deep learning facial recognition system called the
“DeepFace” has been developed by a group of researchers
in the Facebook, which identifies human faces in a digital
image very effectively. Google uses its own facial
recognition system in Google Photos, which automatically
segregates all the photos based on the person in the
image. There are various components involved in Facial
Recognition like the eyes, nose, mouth and the eyebrows.
Self Driving Cars:

Self-driving cars are the Future, there’s no doubt in

that. But the working behind it is very tricky as it
combines a variety of techniques to perceive their
surroundings, including radar, laser light, GPS,
odometry, and computer vision.
Advanced control systems interpret sensory
information to identify appropriate navigation
paths, as well as obstacles and once the image
sensor detects any sign of a living being in its path,
it automatically stops. This happens at a very fast
rate and is a big step towards Driverless Cars.
Security: Object Detection plays a very important role in Security. Be it face ID of Apple or
the retina scan used in all the sci-fi movies.
It is also used by the government to access the security feed and match it with
their existing database to find any criminals or to detect the robbers’ vehicle.
The applications are limitless.
 Links

▪ https://towardsdatascience.com/how-to-train-your-own-object-detector-with-
tensorflows-object-detector-api-bec72ecfe1d9
▪ https://www.kdnuggets.com/2017/10/deep-learning-object-detection-
comprehensive-review.html
▪ http://www.machinelearninguru.com/deep_learning/tensorflow/basics/tfrecord/tfreco
rd.html
▪ https://www.coursera.org/learn/convolutional-neural-networks
▪ https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-
detection-c1f3d437b852
▪ https://towardsdatascience.com/evolution-of-object-detection-and-localization-
algorithms-e241021d8bad
▪ https://medium.freecodecamp.org/how-to-play-quidditch-using-the-tensorflow-
object-detection-api-b0742b99065d
ANY QUERIES!