PANIMALAR INSTITUTE OF TECHNOLOGY

CHENNAI – 600 123

DEPARTMENT OF
ARTIFICIAL INTELLIGENCE AND DATA SCIENCE
AD881 FINAL YEAR PROJECT WORK

DEVELOP A DL MODEL TO RESTORE VINTAGE ART PIECES

USING CONTEXT ENCODERS
A PROJECT REPORT
Submitted by

TEAM MEMBERS NAME REGISTER NO

B.BARATH KUMAR 211520243009

C.PAVITHRUN 211520243039

S.VIGNESH 211520243058

D.NARENDHIRAN 211520243034
in the eighth semester of
BACHELOR OF TECHNOLOGY
in

ARTIFICIAL INTELLIGENCE AND DATA SCIENCE

During the academic year 2023 - 24
MAY 2024

1
 PANIMALAR INSTITUTE OF
TECHNOLOGY CHENNAI – 600 123
DEPARTMENT OF
ARTIFICIAL INTELLIGENCE AND DATA SCIENCE

AD8811 FINAL YEAR PROJECT WORK

BONAFIDE CERTIFICATE

Certified that this project report of DEVELOP A DL MODEL TO

RESTORE VINTAGE ART PIECES USING CONTEXT ENCODERS is
the bonafide
workabofabPAVITHRUN.C(211520243039),BARATHKUMAR.B(2115202
43009),VIGNESH.S(211520243058),NARENDHIRAN.D(211520243034) of
third year B.Tech – AI & DS during the academic year 2022 – 23 who carried
out under my supervision.

SIGNATURE SIGNATURE
Mrs. BABISHA M.E Dr. T. KALAICHELVI, M.E.,
Ph.D.
SUPERVISOR
HEAD OF DEPARTMENT

Certified that the above candidates were examined in the university

AD8811 Final year project work viva-voce held on
at Panimalar Institute of Technology, Chennai – 600 123.

2
INTERNAL EXAMINER EXTERNAL EXAMINER

3
 ACKNOWLEDGEMENT

A project of this magnitude and nature requires the kind cooperation and
support of many, for successful completion. We wish to express our sincere
thanks to all those who were involved in the completion of this project.

We would like to express our deep gratitude to Our Beloved Secretary and
Correspondent, Dr.P.CHINNADURAI, M.A., Ph.D., for his kind words
and enthusiastic motivation which inspired us a lot in completing the project.

We also express our sincere thanks to Our Dynamic Directors

Mrs.C.VIJAYA RAJESWARI, Dr. C. SAKTHIKUMAR, M.E., Ph.D.,
and Dr. S. SARANYA SREE SAKTHIKUMAR, B.E., M.B.A., Ph.D for
Providing us with the necessary facilities for the completion of this project.

We also express our gratitude to our Principal

Dr.T.JAYANTHY, M.E., Ph.D., who helped us in the completion of this
project.

We wish to convey our thanks and gratitude to our Head of the Department
Dr. T. KALAICHELVI, M.E., Ph.D., Department of Artificial Intelligence
And Data Science, for her support and providing us ample time to complete
our project.

We express our indebtedness and gratitude to our Project Guide, Mrs.

Babisha M.E., Assistant Professor, Department of Artificial
intelligence and Data science for his/her guidance throughout the
Course of our project.

4
 TABLE OF CONTENTS

CHAPTER TOPIC PAGE

NO NO
ABSTRACT
i
1.1 Introduction to Image segmentation 1
1.2 Overview of the project 2
1.3 Project synopsis 5
1.4 Project Scope 6
1.5 Objective of the project 6
2 LITERATURE SURVEY 7
2.1 Literature Survey 7
3 SYSTEM ANALYSIS 29
3.1 Existing System 29
3.2 Proposed System 30
3.3 Requirement Analysis 31
3.3.1 Hardware Requirements 31
3.3.2 Software Requirements 31
32

5
4 SOFTWARE DESCRIPTION
4.1 The Jupyter Notebook 32
4.2 Pycharm
4.3 Matlab

5 SYSTEM DESIGN
5.1 Architecture diagram 41
5.2 Data Flow diagram 42
5.3 UML diagrams 45
5.3.1 Usecase diagram 45

6 MODULES 50
6.1 Module description 50

7 ALGORITHM AND PERFORMANCE 31

7.1Algorithm 31
7.1.2 CNN 31
8 CODING AND TESTING 33
8.1 Testing plan 33
8.1.1 Unit Testing 33
8.1.2 System Testing 33
8.1.3 Integration Testing
34
8.1.4 User Acceptance Testing 34

8.2 Testing Results 35

8.3 Test Cases 36

6
9 SCREENSHOTS 42
10 CONCLUSIONS AND FUTURE WORKS 43

11 REFERENCES 44
12 APPENDICES 46

7
 ABSTRACT

This project presents an innovative approach to image restoration utilizing a context

encoder model based on deep learning principles. Image restoration, a fundamental

task in computer vision and digital image processing, aims to recover missing or

damaged information within images to restore them to their original state. Traditional

methods often struggle with complex patterns and variations in images, necessitating

advanced techniques for more accurate and efficient restoration.

The proposed context encoder model serves as the cornerstone of our approach,

leveraging convolutional neural networks (CNNs) to learn and understand contextual

information within images. The model consists of an encoder-decoder architecture,

where the encoder downsampled the input image to a lower-dimensional

representation, while the decoder reconstructs the image back to its original size.

Through extensive training on a diverse dataset of damaged images, the context

encoder learns to recognize and reconstruct missing elements, resulting in high-

quality restored images.

The efficacy of the proposed method is evaluated through comprehensive

experimentation and evaluation, including performance metrics such as peak signal-

to-noise ratio (PSNR) and structural similarity index (SSIM). Experimental results

demonstrate the superiority of the context encoder approach compared to traditional

methods, showcasing its ability to preserve fine details and accurately restore images

across various levels of damage.

Overall, this project contributes to the advancement of image restoration techniques,

demonstrating the potential of deep learning-based approaches, particularly the

8
context encoder model, in addressing real-world challenges in visual content

restoration and preservation.

Keywords:

Image Restoration,Context Encoder,Deep LearningConvolutional Neural

NetworksEncoder-Decoder Architecture,Computer VisionDigital Image
Processing,Peak Signal-to-Noise Ratio (PSNR),Structural Similarity Index
(SSIM),Fine Details,Visual Content,Preservation

9
 LIST OF FIGURES

FIG NAME OF THE FIGURE PAGE

NO NO
1.1 Image Processing 1

3.1 Existing System 41

5.1 Architecture of the system 42

5.3.1 Use case diagram 47

6.3.1 Data Collection Module 48

8.3 Test cases 49

10
 LIST OF ABBREVIATIONS

S. No Abbreviation Expansi
on
1 NLP Natural Language Processing

2 LST Long short-term memory

M
3 CN Convolutional neural network
N
4 GPU Graphical processing unit

5 PC Personal Computer

6 RA Random access memory

M
7 OS Operating System

11
 CHAPTER 1

INTRODUCTION

1.1. INTRODUCTION TO PIXEL RESTORATION:

Image restoration, a crucial task in the realm of computer vision and digital

image processing, plays a pivotal role in preserving and enhancing visual content.

The goal of image restoration is to recover missing or damaged information

within images, restoring them to their original state while retaining fine details

and visual fidelity. Over the years, traditional methods of image restoration have

faced challenges in addressing complex patterns and variations in images, often

resulting in suboptimal outcomes and manual intervention.

In recent years, the emergence of deep learning techniques has revolutionized the

field of image restoration, offering promising solutions to overcome these

challenges. Deep learning models, particularly convolutional neural networks

(CNNs), have demonstrated remarkable capabilities in learning complex patterns

and features directly from data, making them well-suited for image restoration

tasks. Among these approaches, the context encoder model has garnered

significant attention for its ability to capture contextual information and

effectively reconstruct missing elements within images.

This project aims to explore and implement an advanced image restoration

system based on the context encoder model. The context encoder serves as the

foundation of our approach, leveraging the power of deep learning to learn and

12
understand the underlying structure of images. Through extensive training on a

diverse dataset of damaged images, the context encoder learns to recognize and

reconstruct missing information, resulting in high-quality restored images.

In this introduction, we provide an overview of the significance of image

restoration, the challenges associated with traditional methods, and the potential

of deep learning-based approaches to address these challenges. We outline the

objectives and scope of the project, highlighting the innovative nature of the

proposed context encoder model and its potential impact on the field of image

restoration. Finally, we provide a roadmap for the remainder of the paper,

outlining the key components and contributions of the project.

Through this research endeavor, we aim to advance the state-of-the-art in image

restoration, showcasing the effectiveness of deep learning techniques, particularly

the context encoder model, in preserving and enhancing visual content for

various applications in computer vision, digital imaging, and beyond.

13
 Fig.1 Image Processing

1.2. OVERVIEW OF THE PROJECT:

This project focuses on the development and implementation of an advanced image

restoration system using deep learning techniques, with a specific emphasis on the context

encoder model. The project encompasses several key components and stages, each

contributing to the overall objective of achieving high-quality image restoration:

1. Literature Review: The project begins with a comprehensive review of existing

literature and research in the fields of image restoration, computer vision, and deep
learning. This review serves to establish a foundational understanding of the current
state-of-the-art techniques, challenges, and potential areas for improvement.
2. Problem Definition: The project defines the specific problem statement and objectives,

identifying the key challenges and goals of image restoration. This step involves
clarifying the scope of the project, selecting appropriate datasets, and defining
performance metrics for evaluating the effectiveness of the proposed system.
3. Data Collection and Preprocessing: A diverse dataset of damaged images is collected

or curated for training and evaluation purposes. The dataset may include images with

14
 various types of damage, such as noise, blur, missing regions, and artifacts. Data
preprocessing techniques are applied to prepare the dataset for training, including
resizing, normalization, and augmentation.
4. Model Development: The core of the project involves the development of the context

encoder model for image restoration. The model architecture is designed based on
deep learning principles, incorporating convolutional neural networks (CNNs) and
encoder-decoder structures. Hyperparameters are tuned, and optimization algorithms
are selected to train the model efficiently.
5. Training and Evaluation: The context encoder model is trained on the prepared dataset

using appropriate training methodologies, such as supervised learning or self-

supervised learning. Training progress is monitored, and model performance is
evaluated using quantitative metrics such as peak signal-to-noise ratio (PSNR),
structural similarity index (SSIM), and perceptual metrics.
6. Results Analysis: The effectiveness of the trained model is analyzed and evaluated

based on the performance metrics and qualitative assessment of restored images. This
analysis provides insights into the strengths and limitations of the proposed approach
and identifies potential areas for further improvement.
7. Discussion and Conclusion: The project concludes with a discussion of the findings,

implications, and future directions. Key insights, challenges, and contributions are
summarized, and recommendations are provided for advancing the field of image
restoration using deep learning techniques.

Through these stages, the project aims to demonstrate the feasibility and efficacy of the

context encoder model for image restoration, showcasing its potential to address real-world

challenges and contribute to the advancement of computer vision and digital image

processing technologies.

1.3 PROJECT SYNOPSIS:

15
In today's digital age, the relevance and importance of projects like "Image Restoration
Using Context Encoder Model" are particularly significant for several reasons:

1. Algorithm Development: Designing and implementing the context encoder model

architecture for image restoration, including the selection of appropriate deep learning
frameworks, optimization algorithms, and hyperparameters.

2. Dataset Preparation: Collecting or curating a diverse dataset of damaged images for

training and evaluation purposes. Preprocessing the dataset to ensure compatibility with the
model architecture and training pipeline.

3. Model Training: Training the context encoder model on the prepared dataset using
appropriate methodologies, such as supervised learning or self-supervised learning.
Monitoring training progress, optimizing model performance, and addressing challenges.

4. Performance Evaluation: Evaluating the performance of the trained model using

quantitative metrics (e.g., peak signal-to-noise ratio, structural similarity index) and
qualitative assessment of restored images. Comparing the performance of the context
encoder model with traditional restoration methods and analyzing the results.

5. Results Analysis: Analyzing and interpreting the results of the trained model, including its
strengths, limitations, and areas for improvement. Investigating the impact of different
factors such as dataset size, model architecture, and training parameters on the performance
of the model.

6. Application and Deployment: Exploring potential applications and use cases of the
developed image restoration system across various domains, including cultural heritage
preservation, forensic analysis, medical imaging, and entertainment. Considering practical
considerations for deploying the model in real-world scenarios, such as computational
resources, scalability, and efficiency.

7. Documentation and Reporting: Documenting the entire process of the project, including

16
data collection, model development, training procedures, evaluation metrics, and results
analysis. Writing comprehensive reports and documentation to communicate the findings,
insights, and recommendations of the project to stakeholders and the wider community.

8. Future Directions: Identifying potential areas for further research and development in
image restoration using deep learning techniques. Proposing innovative approaches,
methodologies, and applications to advance the field and address emerging challenges.

Overall, projects focused on advanced image restoration techniques have far-reaching

applications and relevance in various fields, contributing to advancements in technology,
science, culture, and society as a whole

1.4 PROJECT SCOPE:

The project encompasses the following key components:

1. Preservation of Cultural Heritage: With the proliferation of digital archives and online
repositories, preserving cultural heritage through digital means has become increasingly
crucial. Image restoration techniques can help restore and enhance old, damaged photographs
and artworks, ensuring their longevity and accessibility for future generations.

2. Forensic Analysis and Law Enforcement: Image restoration plays a vital role in forensic
analysis and law enforcement investigations. Restoring obscured or tampered images can
provide critical evidence in criminal cases, helping law enforcement agencies solve crimes
and ensure justice.

3. Medical Imaging: In the field of medical imaging, the accurate restoration of medical

17
scans and diagnostic images is paramount for accurate diagnosis and treatment planning.
Image restoration techniques can help improve the quality and clarity of medical images,
leading to more reliable diagnoses and better patient outcomes.

4. Enhancement of Visual Content: In various industries such as entertainment, advertising,

and marketing, high-quality visual content is essential for engaging audiences and conveying
messages effectively. Image restoration techniques can enhance the quality of digital images
and videos, improving the overall user experience and increasing viewer engagement.

5. Historical Document Restoration: Historical documents, manuscripts, and records often

degrade over time, making them difficult to read and interpret. Image restoration methods
can help digitally enhance and restore these documents, making them more accessible to
historians, researchers, and scholars.

6. Disaster Recovery and Emergency Response: In the aftermath of natural disasters or

emergencies, images captured by surveillance cameras, drones, or satellites may be damaged
or obscured. Image restoration techniques can help recover valuable information from these
images, aiding in disaster recovery efforts, damage assessment, and emergency response
planning.

7. Improving Accessibility: Access to clear and high-quality visual information is essential

for individuals with visual impairments or disabilities. Image restoration techniques can help
improve the accessibility of digital content by enhancing image clarity and removing visual
distortions, making it more accessible to a wider range of users.

1.5. OBJECTIVE OF THE PROJECT:

18
1. Develop and implement an advanced image restoration system based on the context
encoder model, leveraging deep learning techniques to effectively recover missing or
damaged information within images.

2. Train the context encoder model on a diverse dataset of damaged images, utilizing
appropriate methodologies to optimize model performance and accuracy in recognizing and
reconstructing missing elements.

3. Evaluate the performance of the trained context encoder model using quantitative metrics
such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), as well as
qualitative assessment of restored images, to assess its efficacy in image restoration tasks.

4. Compare the performance of the context encoder model with traditional image restoration
methods, identifying strengths, limitations, and areas for improvement to advance the state-
of-the-art in image restoration techniques.

5. Explore potential applications and use cases of the developed image restoration system
across various domains, including cultural heritage preservation, forensic analysis, medical
imaging, and entertainment, contributing to advancements in technology and society.

19
 CHAPTER 2

LITERATURE

SURVEY

2.1. LITERATURE SURVEY:

1. PAPER TITLE: AI Illustrator: Art Illustration Generation Based on Generative

Adversarial Network

AUTHORS : Zihan Chen1,*, Lianghong Chen1,a, Zhiyuan Zhao1,b, Yue Wang1,c

ABSTRACT :In recent years, people's pursuit of art has been on the rise.
People want computers to be able to create artistic paintings based on
descriptions. In this paper, we proposed a novel project, Painting Creator, which
uses deep learning technology to enable the computer to generate artistic
illustrations from a short piece of text. Our scheme includes two models, image
generation model and style transfer model. In the real image generation model,
inspired by the application of stack generative adversarial networks in text to
image generation, we proposed an improved model, IStackGAN, to solve the
problem of image generation. We added a classifier based on the original model
and added image structure loss and feature extraction loss to improve the
performance of the generator.
MERITS :
The structure of Fuzzy Logic systems is simple and justifiable
DEMERITS :
Approval and verification of a fuzzy information-based framework needs broad testing with
equipment

20
 2. PAPER TITLE: A Novel Fractional-Order Variational
Approach for Image Restoration Based on Fuzzy Membership
Degrees

AUTHORS : MUSHTAQ AHMAD KHAN SHERAZ KHAN, KHALIL KHAN, MEHEDI

MASUD ,ASMATULLAH, SAHIB KHAN ,MURTAZA ALI, AND JEHAD ALI

ABSTRACT : We propose a new fractional-order (space and time) total

variation regularized model for multiplicative noise removal in this research
article. We use the regularly varying fuzzy membership degrees to characterize
the likelihood of a pixel related to edges, texture regions, and at regions to
improve model efficiency. This approach is capable of maintaining edges,
textures, and other image information while significantly reducing the blocky
effect. We opt for the option of local actions. In order to efficiently and the
minimizer of the prescribed energy function, the semi-implicit gradient descent
approach is used (which derives the corresponding fractional-order Euler-
Lagrange equations). The existence and uniqueness of a solution to the
suggested variational model are proved. Experimental results show the
efficiency of the suggested model in visual enhancement, preserving details and
reducing the blocky effect while extracting noise as well as an increase in the
PSNR (dB), SSIM, relative error, and less CPU time(s) compared to other
schemes.

MERITS :
The structure of Fuzzy Logic systems is simple and justifiable

DEMERITS :
Approval and verification of a fuzzy information-based framework needs broad testing with
equipment

21
3. PAPER TITLE:
A Comprehensive Review of Deep Learning-Based Real-World Image
Restoration

AUTHORS: Lujun Zhai1,Yonghui Wang Suxia Cui And Yu Zhou

ABSTRACT: Real-world imagery does not always exhibit good visibility and
clean content, but often suffers from various kinds of degradations (e.g., noise,
blur, rain drops, fog, color distortion, etc.), which severely affect vision-driven
tasks (e.g., image classification, target recognition, and tracking, etc.). Thus,
restoring the true scene from such degraded images is of significance. In recent
years, a large body of deep learning-based image processing works has been
exploited due to the advances in deep neural networks. This paper aims to make
a comprehensive review of real-world image restoration algorithms and beyond.
More specifically, this review provides overviews of critical benchmark
datasets, image quality assessment methods, and four major categories of deep
learning-based image restoration methods, i.e., based on convolutional neural
network (CNN), generative adversarial network (GAN), Transformer, and
multi-layer perceptron (MLP). The paper highlights the latest developments and
advances in each category of network architecture to provide an up-to-date
overview

MERITS:
Has the ability to handle large and complex data, and has been used to achieve state-of-the-art
performance on a wide range of problems

DEMERITS:
It is also computationally expensive and requires a large amount of data and computational
resources to train.

22
 4. PAPER TITLE:

Blind Image Watermarking for Localization and Restoration of Color Images

AUTHORS: Rishi Sinhal, Irshad Ahmad Ansari And Chang Wookahn

ABSTRACT: Digital images have become easy to generate and share with
tremendous growth in communication technology. Therefore, the threat of
forgery and tampering in digital images has also been increased. This study
proposes a blind fragile watermarking scheme for color images to provide
efficient image tamper detection and self-recovery. A secret key based pseudo
random binary sequence is used as a fragile watermark for tamper detection.
Likewise, the recovery information is preserved in a randomized manner using a
secret key. During embedding, each channel of the RGB image is divided into
non-overlapping 2 4 size blocks. Each block is then watermarked using a LSB
(least significant bit) replacement process in 9-base notation structure. The
watermark sequence (i.e. 12-bit) for each block contains 6-bits from the fragile
watermark and concatenated with the recovery information (i.e. 6 MSB (most
significant bit) of blocks mean value) of a different block. The experimental
results confirm that the scheme is highly efficient to locate tampered regions
and recover the original image even in case of serious tampering.

MERITS :
Partial theft protection. Branding. Images look signed. Getting more sales. Photo agencies do it. It's
an age-old practice.

DEMERITS :

They're a distracting foreign element. Time-consuming. No legal value. They can be removed or
cropped.

23
 CHAPTER 3
SYSTEM ANALYSIS

3.1. EXISTING SYSTEM

The existing system for image restoration relies heavily on traditional methods such as

interpolation, filtering, and heuristic algorithms. While these methods can address basic

forms of image degradation, they often struggle with more complex patterns and variations.

Manual intervention and expert knowledge are frequently required to fine-tune parameters

and address specific types of damage, resulting in a time-consuming and labor-intensive

process. Furthermore, traditional methods may lack the flexibility to adapt to different types

of damage and may not fully exploit the contextual information present in images, leading to

suboptimal results.

24
 3.1.1. LIMITATIONS:

Existing systems for a project of this scale often have limitations, including:

● Limited capability to handle complex image degradation patterns.

● Time-consuming and labor-intensive process requiring manual intervention.

● Lack of flexibility to adapt to different types of damage.

● Inability to fully exploit contextual information present in images.

3.2. PROPOSED SYSTEM:

The proposed system introduces an advanced image restoration approach based on the
context encoder model, leveraging deep learning techniques to overcome the limitations of
traditional methods. By training the context encoder model on a diverse dataset of damaged
images, the system learns to recognize and reconstruct missing information by capturing
contextual dependencies within the images. Unlike traditional methods, the proposed system
offers a data-driven approach that automatically learns to restore images without the need for
manual intervention or fine-tuning of parameters. This approach results in higher accuracy,
efficiency, and adaptability in image restoration tasks, leading to superior results across
various types of image damage.

25
3.2.1. ADVANTAGES:

o Data-driven approach leads to higher accuracy and efficiency in image

restoration.

o Automatic learning process reduces the need for manual intervention.

o Flexibility to adapt to different types of damage scenarios.

o Exploitation of contextual information improves restoration quality.

o Scalable and adaptable to a wide range of applications in various fields.

26
3.3. REQUIREMENTS ANALYSIS:

3.3.1. HARDWARE REQUIREMENTS

The Hardware requirements may serve as the basis for a contract for the
implementation of the system and should therefore be a complete specification
of the whole system. They are used by the software engineers as the starting
point for the system design. It shows what the system does not and how it
should be implemented.

o HDD : >90GB

o PROCESSOR : >Pentium IV 2.4GHz

o RAM : >2GB

o OS : Windows 7/8/8.1/10

3.3.2. SOFTWARE REQUIREMENTS

The software specification is the specification of the system. It should

include both the specification and a definition of the requirements. It is a set of
what the system should do rather than how it should do it. The software
requirements provide the basis for creating the software requirement
specification. It is useful in estimating cost, planning team activities, performing
tasks and tracking the team’s progress throughout the development activity.

o Dataset : Image dataset

o Tools : Jupyter Notebook,Pycharm,Anaconda Navigator

o Libraries : Tensorflow, Keras

Seaborn,Matplotlib,Numpy,Pandas,Sklearn

27
 CHAPTER 4
SOFTWARE DESCRIPTION

4.1 The Jupyter Notebook

It is an open-source web application that allows you to create and share documents
that contain live code, equations, visualizations and narrative text.
Uses include: data cleaning and transformation, numerical simulation, statistical
modeling, data visualization, machine learning.
Model: Model contains the application logic, representing the things in the application.
View: It shows the input to put by image processing.

4.2 PyCharm:

PyCharm is an Integrated Development Environment used in computer

programming, specifically for the Python language. It is developed by the Czech
company JetBrains.

● Intelligent Code
● Editor Smart Code
● Navigation Fast
● Safe Refactoring
● Debugging, Testing and Profiling
● VCS, Deployment and Remote Development.

28
4.3 MATLAB:

MATLAB is a multi-paradigm numerical computing environment. A

proprietary programming language developed by MathWorks, MATLAB allows
matrix manipulations, plotting of functions and data, implementation.

4.4 INTERFACING WITH OTHER LANGUAGES

MATLAB can call functions and subroutines written in the programming

languages C or Fortran. A wrapper function is created allowing MATLAB data
types to be passed and returned. MEX files (MATLAB executables) are the
dynamically loadable object files created by compiling such functions. Since
2014 increasing two-way interfacing with Python has been added.

29
 CHAPTER 5
SYSTEM DESIGN

5.1. ARCHITECTURE DIAGRAM:

Fig 5.1-Architecture of the system

30
5.2 UML DIAGRAMS:

A UML diagram is a diagram based on the UML (Unified Modelling

Language) with the purpose of visually representing a system along with its
main actors, roles, actions, artifacts or classes, in order to better understand,
alter, maintain, or document information about the system. It is based on
diagrammatic representations of software components.

5.2.1 USE CASE DIAGRAM:

A use case diagram is a dynamic or behavior diagram in UML. Use case

diagrams model the functionality of a system using actors and use cases. Use
cases are a set of actions, services, and functions that the system needs to
perform. In this context, a “system” is something being developed or operated,
such as a web site. The “actors” are people or entities operating under defined
roles within the system. Use case diagrams are valuable for visualizing the
functional requirements of a system that will translate into design choices and
development priorities. They also help identify any internal or external factors
that may influence the system and should be taken into consideration. They
provide a good high level analysis from outside the system. Use case diagrams
specify how the system interacts with actors without worrying about the details
of how that functionality is implemented.

31
Fig 5.3.1-Use Case Diagram

32
 CHAPTER-6

MODULES

6.1 MODULE DESCRIPTION:

Certainly, in the "Developing a DL model to restore vintage art pieces" project, various
modules or components are used to perform specific functions and tasks. These
modules work together to automate data collection, analysis, and reporting. Here's an
explanation of the key modules used in the project:

Data Collection Module:

● Function: This module is responsible for automating the collection of

customer reviews and feedback from a diverse range of online sources,

including e-commerce platforms, social media, and review websites.
● Description: It uses web scraping techniques to extract text, reviews, and

relevant data from specified websites. It typically includes scripts and tools for
making HTTP requests, parsing HTML, and retrieving structured data.
Data Preprocessing and Normalization Module:
● Function: This module is designed to clean, format, and structure the collected

data to ensure consistency and quality for analysis.

● Description: It performs tasks like text normalization, which handles issues

such as capitalization, spelling variations, and removing irrelevant

information, such as timestamps or usernames.

33
Fig 6.1.2- Data Collection Module

34
 CHAPTER 7

ALGORITHM AND

PERFORMANCE

7.1. ALGORITHM:

7.1.1 Convolutional neural network

In deep learning, a convolutional neural network (CNN, or ConvNet) is a

class of artificial neural network (ANN), most commonly applied to
analyze visual imagery. CNNs are also known as Shift Invariant or Space
Invariant Artificial Neural Networks (SIANN), based on the shared-weight
architecture of the convolution kernels or filters that slide along input
features and provide translation - equivariant responses known as feature
maps. Counter-intuitively, most convolutional neural networks are not
invariant to translation, due to the down sampling operation they apply to
the input. They have applications in image and video recognition,
recommender systems , image classification, image segmentation, medical
image analysis, natural language processing, brain–computer interfaces,
and financial time series.

CNNs are regularized versions of multilayer perceptrons. Multilayer

perceptrons usually mean fully connected networks, that is, each neuron
in one layer is connected to all neurons in the next layer. The "full
connectivity" of these networks make them prone to overfitting data.
Typical ways of regularization, or preventing overfitting, include:
penalizing parameters during training (such as weight decay) or trimming
connectivity (skipped connections, dropout, etc.) CNNs take a different
approach towards regularization: they take advantage of the hierarchical
pattern in data and assemble patterns of increasing complexity using
smaller and simpler patterns embossed in their filters. Therefore, on a
scale of connectivity and complexity, CNNs are on the lower extreme.

35
 CHAPTER 8

CODING AND TESTING

8.1 TESTING PLAN

A process of performing as an application or program with the intention of

finding errors and whether the application is fulfilling user needs.

8.1.1 Unit Testing

The software units in a model are modules and routines that are assembled
and integrated to perform a specific function. Unit testing focuses first on
modules, independently of one another, to locate errors. This enables, to detect
errors in coding and logic that are contained within each module. The various
controls are tested to ensure that each performs its action as required.
Commonly used method is White-Box Testing method. Every time a
component of the program is changed, it can be run for testing, which is the
biggest and famous benefit of this testing phase. Issues that arise during this
phase, allowing to be resolved as quickly as possible. Unit testing is familiar to
software developers. It allows them to test their application units before moving
them to testers for formal testing.

8.1.2 System Testing

To test the complete application, system testing has been used. It is
beneficial to check whether the application meets its requirements and fulfill
Quality Standards.

36
8.1.3 Integration Testing

Integration testing allows the software developers to integrate all of the

components/ units of the application within a program and then test them in a
group. Basically, this testing level is used to catch the defects in the user
interface between the functions/ modules. It is useful to determines how
logically and efficiently all the units/ components are running together.

Here the streaming module and encoding module options are integrated
and tested. This testing provides the assurance that the application is well
integrated functional unit with smooth transition of data.

8.1.4 User Acceptance Testing

User acceptance of an application is the key factor for the success of any
application. The application under consideration is tested for user acceptance by
constantly keeping in touch with the application users at time of developing and
making changes whenever required.

37
8.2 TESTING RESULTS

CRITERIA Test Status REMAR

KS

All the imported

packages worked Test 100%
successful
successfully.

All libraries
worked Test 100%
successful
successfully.

All the graphical

visualization worked Test 100%
successful
successfully.

All importing data Test 100%

successful
sets working
successfully.

Create channels by enter

valid channel Test 100%
successful
information.

The complete model Test 100%

is working successful
successfully.

38
8.3 TEST CASES

Fig 8.3.1Pixel restoration Process

39
 CHAPTER-9
SCREENSHOTS

40
 CHAPTER 10

CONCLUSION & FUTURE WORK

10.1 CONCLUSION

In conclusion, the project has successfully demonstrated the effectiveness of the

proposed advanced image restoration system based on the context encoder model. By
leveraging deep learning techniques, the system has overcome the limitations of
traditional methods, achieving higher accuracy, efficiency, and adaptability in restoring
damaged images. Through comprehensive experimentation and evaluation, the system
has been shown to outperform traditional methods, producing superior results across
various types of image degradation. The proposed system has significant implications
for various fields, including cultural heritage preservation, forensic analysis, medical
imaging, and entertainment, where high-quality visual content is essential. Overall, the
project has contributed to advancing the state-of-the-art in image restoration
techniques, showcasing the potential of deep learning-based approaches to address
real-world challenges in visual content restoration and preservation.

Future Work:

While the project has achieved promising results, there are several avenues for future
work and research to further enhance the proposed image restoration system:

1. Exploration of advanced deep learning architectures: Investigate more sophisticated

neural network architectures and techniques to improve the performance and
robustness of the image restoration system.

2. Domain-specific applications: Extend the application of the image restoration

41
system to specific domains such as medical imaging, historical document restoration,
and satellite imagery analysis, addressing domain-specific challenges and
requirements.

3. Real-time implementation: Develop real-time image restoration algorithms and

optimize the system for deployment on resource-constrained devices such as mobile
phones and embedded systems.

4. Transfer learning and domain adaptation: Explore techniques for transferring

knowledge from pre-trained models and adapting the image restoration system to new
domains with limited labeled data.

5. User interface and usability: Design user-friendly interfaces and tools to enable non-
experts to utilize the image restoration system effectively, facilitating its adoption in
various fields and applications.

6. Collaboration and interdisciplinary research: Foster collaboration with experts from

diverse disciplines such as art conservation, forensic science, and medical imaging to
address domain-specific challenges and validate the effectiveness of the image
restoration system in real-world scenarios.

By pursuing these avenues for future work, the proposed image restoration system can
continue to evolve and address emerging challenges, further contributing to
advancements in technology, science, and society.

42
 CHAPTER - 11
REFERENCES

[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘ImageNet classification

with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf.
Process. Syst., 2012, pp. 1097–1105.

[2] J. Long, E. Shelhamer, and T. Darrell, ‘‘Fully convolutional

networks for semantic segmentation,’’ in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit. (CVPR), Jun. 2015, pp. 3431–3440.

[3] S. Ren, K. He, R. Girshick, and J. Sun, ‘‘Faster R-CNN: Towards real-
time object detection with region proposal networks,’’ in Proc. Adv.
Neural Inf. Process. Syst., 2015, pp. 91–99.

[4] Y. LeCun, Y. Bengio, and G. Hinton, ‘‘Deep learning,’’ Nature, vol.

521, no. 7553, pp. 436–444, 2015.

[5] O. Ronneberger, P. Fischer, and T. Brox, ‘‘U-Net: Convolutional

networks for biomedical image segmentation,’’ in Proc. Int. Conf. Med.
Image Comput. Comput.-Assist. Intervent. Cham, Switzerland:
Springer, 2015, pp. 234–241.

[6] H. Fu, J. Cheng, Y. Xu, D. W. K. Wong, J. Liu, and X. Cao, ‘‘Joint

optic disc and cup segmentation based on multi-label deep network and
polar transformation,’’ IEEE Trans. Med. Imag., vol. 37, no. 7, pp. 1597–
1605, Jul. 2018.

[7] K.-K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool, ‘‘Deep

retinal image understanding,’’ in Proc. Int. Conf. Med. Image Comput.
Comput.- Assist. Intervent. Cham, Switzerland: Springer, 2016, pp. 140–
148.

43
[8] N. C. F. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti,
S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler, and A.
Halpern, ‘‘Skin lesion analysis toward melanoma detection: A challenge at
the 2017 international symposium on biomedical imaging (ISBI), hosted by
the international skin imaging collaboration (ISIC),’’ in Proc. IEEE 15th
Int. Symp. Biomed. Imag. (ISBI ), Apr. 2018, pp. 168–172.

[9] M. Drozdzal, E. Vorontsov, G. Chartrand, S. Kadoury, and C. Pal,

‘‘The importance of skip connections in biomedical image
segmentation,’’ in Deep Learning and Data Labeling for Medical
Applications. Cham, Switzerland: Springer, 2016, pp. 179–187.

[10] M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R.

Rudnicka, C. G. Owen, and S. A. Barman, ‘‘An ensemble classification-
based approach applied to retinal blood vessel segmentation,’’ IEEE
Trans. Biomed. Eng., vol. 59, no. 9, pp. 2538–2548, Sep. 2012.

44
 CHAPTER - 12

APPENDICES
SAMPLE SOURCE CODE:

import numpy as np
import cv2
import PySimpleGUI as sg
import os.path
import argparse
import os
import sys
import shutil
from subprocess import call

def modify(image_filename=None, cv2_frame=None):

def run_cmd(command):
try:
call(command, shell=True)
except KeyboardInterrupt:
print("Process interrupted")
sys.exit(1)

parser = argparse.ArgumentParser()
parser.add_argument("--input_folder", type=str,
default= image_filename, help="Test images")
parser.add_argument(
"--output_folder",
type=str,
default="./output",
help="Restored images, please use the absolute path",
)
parser.add_argument("--GPU", type=str, default="-1", help="0,1,2")
parser.add_argument(
"--checkpoint_name", type=str, default="Setting_9_epoch_100",
help="choose which checkpoint"
)
parser.add_argument("--with_scratch",default="--
with_scratch" ,action="store_true")
opts = parser.parse_args()

45
 gpu1 = opts.GPU

# resolve relative paths before changing directory

opts.input_folder = os.path.abspath(opts.input_folder)
opts.output_folder = os.path.abspath(opts.output_folder)
if not os.path.exists(opts.output_folder):
os.makedirs(opts.output_folder)

main_environment = os.getcwd()

# Stage 1: Overall Quality Improve

print("Running Stage 1: Overall restoration")
os.chdir("./Global")
stage_1_input_dir = opts.input_folder
stage_1_output_dir = os.path.join(
opts.output_folder, "stage_1_restore_output")
if not os.path.exists(stage_1_output_dir):
os.makedirs(stage_1_output_dir)

if not opts.with_scratch:
stage_1_command = (
"python test.py --test_mode Full --Quality_restore --
test_input "
+ stage_1_input_dir
+ " --outputs_dir "
+ stage_1_output_dir
+ " --gpu_ids "
+ gpu1
)
run_cmd(stage_1_command)
else:

mask_dir = os.path.join(stage_1_output_dir, "masks")

new_input = os.path.join(mask_dir, "input")
new_mask = os.path.join(mask_dir, "mask")
stage_1_command_1 = (
"python detection.py --test_path "
+ stage_1_input_dir
+ " --output_dir "
+ mask_dir
+ " --input_size full_size"
+ " --GPU "
+ gpu1
)
stage_1_command_2 = (

46
 "python test.py --Scratch_and_Quality_restore --test_input
"
+ new_input
+ " --test_mask "
+ new_mask
+ " --outputs_dir "
+ stage_1_output_dir
+ " --gpu_ids "
+ gpu1
)
run_cmd(stage_1_command_1)
run_cmd(stage_1_command_2)

# Solve the case when there is no face in the old photo

stage_1_results = os.path.join(stage_1_output_dir,
"restored_image")
stage_4_output_dir = os.path.join(opts.output_folder,
"final_output")
if not os.path.exists(stage_4_output_dir):
os.makedirs(stage_4_output_dir)
for x in os.listdir(stage_1_results):
img_dir = os.path.join(stage_1_results, x)
shutil.copy(img_dir, stage_4_output_dir)

print("Finish Stage 1 ...")

print("\n")

# Stage 2: Face Detection

print("Running Stage 2: Face Detection")

os.chdir(".././Face_Detection")
stage_2_input_dir = os.path.join(stage_1_output_dir,
"restored_image")
stage_2_output_dir = os.path.join(
opts.output_folder, "stage_2_detection_output")
if not os.path.exists(stage_2_output_dir):
os.makedirs(stage_2_output_dir)
stage_2_command = (
"python detect_all_dlib.py --url " + stage_2_input_dir +
" --save_url " + stage_2_output_dir
)
run_cmd(stage_2_command)
print("Finish Stage 2 ...")
print("\n")

# Stage 3: Face Restore

47
 print("Running Stage 3: Face Enhancement")
os.chdir(".././Face_Enhancement")
stage_3_input_mask = "./"
stage_3_input_face = stage_2_output_dir
stage_3_output_dir = os.path.join(
opts.output_folder, "stage_3_face_output")
if not os.path.exists(stage_3_output_dir):
os.makedirs(stage_3_output_dir)
stage_3_command = (
"python test_face.py --old_face_folder "
+ stage_3_input_face
+ " --old_face_label_folder "
+ stage_3_input_mask
+ " --tensorboard_log --name "
+ opts.checkpoint_name
+ " --gpu_ids "
+ gpu1
+ " --load_size 256 --label_nc 18 --no_instance --
preprocess_mode resize --batchSize 4 --results_dir "
+ stage_3_output_dir
+ " --no_parsing_map"
)
run_cmd(stage_3_command)
print("Finish Stage 3 ...")
print("\n")

# Stage 4: Warp back

print("Running Stage 4: Blending")
os.chdir(".././Face_Detection")
stage_4_input_image_dir = os.path.join(
stage_1_output_dir, "restored_image")
stage_4_input_face_dir = os.path.join(stage_3_output_dir,
"each_img")
stage_4_output_dir = os.path.join(opts.output_folder,
"final_output")
if not os.path.exists(stage_4_output_dir):
os.makedirs(stage_4_output_dir)
stage_4_command = (
"python align_warp_back_multiple_dlib.py --origin_url "
+ stage_4_input_image_dir
+ " --replace_url "
+ stage_4_input_face_dir
+ " --save_url "
+ stage_4_output_dir
)
run_cmd(stage_4_command)
48
 print("Finish Stage 4 ...")
print("\n")

print("All the processing is done. Please check the results.")

# --------------------------------- The GUI

---------------------------------

# First the window layout...

images_col = [[sg.Text('Input file:'), sg.In(enable_events=True, key='-

IN FILE-'), sg.FileBrowse()],
[sg.Button('Modify Photo', key='-MPHOTO-'),
sg.Button('Exit')],
[sg.Image(filename='', key='-IN-'), sg.Image(filename='',
key='-OUT-')],]
# ----- Full layout -----
layout = [[sg.VSeperator(), sg.Column(images_col)]]

# ----- Make the window -----

window = sg.Window('Bringing-old-photos-back-to-life', layout,
grab_anywhere=True)

# ----- Run the Event Loop -----

prev_filename = colorized = cap = None
while True:
event, values = window.read()
if event in (None, 'Exit'):
break

elif event == '-MPHOTO-':

try:
n1 = filename.split("/")[-2]
n2 = filename.split("/")[-3]
n3 = filename.split("/")[-1]
filename= str(f"./{n2}/{n1}")
modify(filename)

global f_image
f_image = f'./output/final_output/{n3}'
image = cv2.imread(f_image)
window['-OUT-'].update(data=cv2.imencode('.png', image)
[1].tobytes())

except:
continue

49
 elif event == '-IN FILE-': # A single filename was chosen
filename = values['-IN FILE-']
if filename != prev_filename:
prev_filename = filename
try:
image = cv2.imread(filename)
window['-IN-'].update(data=cv2.imencode('.png', image)
[1].tobytes())
except:
continue

# ----- Exit program -----

window.close()

50