A Dissertation work entitled on

Image Colorization using Deep Learning

submitted in partial fulfillment of the requirements for the award of the degree of

Master of Technology
in

Computer Science and Engineering

By

Faria Sanaa Mohammed

160621742108

Under the guidance of

Mrs. M Sowmya

Assistant Professor

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,

Stanley College of Engineering and Technology for Women (Autonomous)

(Approved by AICTE, Accredited by NBA and NAAC, Affiliated to Osmania University)

Chapel Road, Abids, Hyderabad

2023
 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Stanley College of Engineering and Technology for Women (Autonomous)

(Approved by AICTE, Accredited by NBA and NAAC, Affiliated to Osmania University)

CERTIFICATE
This is to certify that the project entitled “Image Colorization using Deep Learning”
submitted by Faria Sanaa Mohammed(160621742108), in partial fulfillment of the
requirements for the award of the degree of Master of Technology in Department of Computer
Science and Engineering to the Stanley College of Engineering and Technology for Women,
Abids, is a record of bonafide work carried out under my guidance and supervision. The results
embodied in this project report have not been submitted to any other University or Institute for
the award of any Degree or Diploma.

Signature of the guide Signature of the Head of the Dept.

Mrs. M Sowmya Dr YVSS Pragathi

Assistant Professor, Professor & HOD,

Dept. of CSE. Dept. of CSE.

External Examiner
 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
Stanley College of Engineering and Technology for Women (Autonomous)
(Approved by AICTE, Accredited by NAAC, Affiliated to Osmania University)

DECLARATION

I declare that the work reported in the thesis entitled “Image Colorization using Deep
Learning” is record of the work done by me in the Department of Computer Science and
Engineering, Stanley College of Engineering and Technology for women, Hyderabad.

No part of the thesis is copied from books / journals / internet and wherever referred the
same has been duly acknowledge in the text. The reported data is based on the project work done
entirely by me and not copied from any other source.

Faria Sanaa Mohammed

(160621742108)
 ACKNOWLEDGEMENT

It is with a sense of gratitude and appreciation that I feel to acknowledge any well-
wishers for their king support and encouragement during the completion of the project.

I thank Dr. Satya Prasad Lanka, the Principal, Stanley College of Engineering and
Technology for women, for his timely cooperation and for providing me all the required
facilities to complete the project successfully.

I would extremely grateful to Dr. Y.V.S.S Pragathi, Head of the department for
providing excellent computing facilities and such a nice atmosphere for completing my
project.

I would like to express my heartfelt gratitude to Dr. Potu Narayana , Coordinator of

M.Tech in the Computer Science & Engineering department and, my internal supervisor
Mrs. M Soumya , for encouraging and guiding me throughout the project . I am highly
indebted to her for her guidance and constant supervision regarding the project as well as
for providing valuable suggestions and continuous support given to me for completing
the project successfully and all those who helped me directly or indirectly during the
course of project. I sincerely thank all of them.

Faria Sanaa Mohammed

(160621742108)
 ABSTRACT

Colorization is the process in which color components are added to the grayscale image.
The information contained in the grey-scale image is limited than that of color image. Thus
adding the color components can provides more insights about its semantics. Coloring of a
gray scale image is extensively chosen for various researches in graphics and computer
vision especially with deep learning methods. The U-net is a type of deep Convolution
Neural Network (CNN) which consists of the down sampling and up sampling paths. They
are connected through the bottleneck and skip connection between the encoder and decoder
which provides the copy and concatenate the feature maps from encoder to decoder. The
input to the network is gray image of size 224×224 and it produced the RGB color image of
same size as input. The Mean Squared Error (MSE) loss function is to distinguished the
quality between the generated color image to its corresponding ground truth image. The
Peak Signal to Noise Ratio (PSNR) is used for the quality measurement of the predicted
color image and the ground truth image.

Keywords: U-Net, CNN, Mean Square Error, Encoder-Decoder ,PSNR

i
 Table of Contents

CHAPTER TITLES PAGE

NO NO

Abstract i
List of Figures ii
List of Tables iv
List of acronyms v

1 INTRODUCTION 1
1.1 Problem Statement and Description 2
1.2 Motivation 2
1.3 Aim and scope 3
1.4 Background and Basics 4
1.5 Organization of Thesis .17
2. LITERATURE SURVEY 18
3. System Architecture 23
3.1 Usage of LAB color space 24
3.2 Module 1: Dataset Collection 25
3.3 Module 2: Data Analysis and Data preprocessing 26
3.4 Module 3: Data visualization and Feature Selection 27
3.5 Module 4: Data Splitting 29
3.6 Module 5: U-Net architecture 31
3.7 Module 6:Evaluating the model using PSNR 33
4 Implementation 34
5 Experimental results 39
6 Conclusion 46
7 Future scope 47
References 48
 LIST OF FIGURES

FIG. NO FIGURE NAME PAGE

NO.
1.1 CNN algorithm 7
1.2 Typical CNN Architecture 7

1.3 Flattening of a 3x3 image matrix into a 9x1 vector 8

1.4 4x4x3 RGB Image 9
1.5 Convoluting a 5x5x1 image with a 3x3x1 kernel to get a 3x3x1 10
convolved feature
1.6 Movement of the Kernel 11
1.7 Convolution operation on a MxNx3 image matrix with a 3x3x3 Kernel 12
1.8 Convolution Operation with Stride Length = 2 12
1.9 SAME padding: 5x5x1 image is padded with 0s to create a 6x6x1 13
image
1.10 3x3 pooling over 5x5 convolved feature 14
1.11 Types of Pooling 14
1.12 Classification — Fully Connected Layer 15
2.1 (a) Overexposed image. (b) Image after color enhancement using 18
histogram normalization.
2.2. Example of Levin’s method 20

3.1 Proposed system working 24

3.2 CIE Lab color space 25
3.3 U-Net Architecture 32
4.1 necessary libraries and define some parameters. 35
4.2 define the UNet architecture with a ResNet-18 encoder 36
4.3 Optimizer and loss function 36
4.4 define the input and output data transformations and create 37
the training and testing data loaders

ii
5.1 loading the dataset 39

5.2 data splitting 39

5.3 training and testing data 40
5.4 pretrained model 40
5.5 images in dataset 41
5.6 (a) After 10 epochs (b) training and testing losses, respectively, 42
for each epoch
5.7 (a) After 20 epochs (b) training and testing losses, respectively, 43
for each epoch
5.8 (a) After 35 epochs (b) training and testing losses, respectively, 44
for each epoch
5.9 Evaluation using PSNR 45

iii
 LIST OF TABLES

TABLE No. CAPTION PAGE

No.
4.1 Frequency of tags in the MIR Flickr set 34

iv
 LIST OF ACRONYMS

Acronyms Full form

CNN convolutional neural networks

RGB Red green blue color space

NLP natural language processing

HSV hue, saturation, value

CMYK cyan, magenta, yellow, and key (black)

SURF speeded up robust features

GAN generative adversarial network

PCA principal component analysis

MSE mean squared error

PSNR peak signal-to-noise ratio

dB decibels
CPU central processing unit
GPU graphics processing unit