CineChrome: Transforming the past into

colourful present
A PrePrint
B Jamin Enock Amrita Dutta
School of Computing and School of Computing and
Information Technology Information Technology
20010516193.JAMINENOCK@cit.reva.edu.in 20010510376.AMRITADUTTA@cit.reva.edu.in

Deon Levon Dmello Anurag Kunde

School of Computing and School of Computing and
Information Technology Information Technology
r20200093.DeonLevon@cit.reva.edu.in r20200103.anurag@cit.reva.edu.in

Abstract
This project delves into the intriguing realm of colorizing black and white images through
the lens of deep learning. Motivated by a desire to breathe vitality into historical
photographs, we embark on a transformative journey to develop a convolutional neural
network (CNN) capable of infusing grayscale images with authentic and contextually
relevant colors.
The motivation stems from the inherent charm of black and white photography, which
distills moments into a narrative focused on form and emotion. However, the absence of
color often leaves historical images feeling detached from the vibrancy of the lived
experience. Our project seeks to address this by not only colorizing images but also
preserving their authenticity and narrative context.
In navigating the extensive literature on image colorization, we uncover a dynamic
evolution from traditional heuristic-based methods to contemporary deep learning
approaches. Drawing insights from this rich tapestry, our project aims to carve a unique
niche by synthesizing the interpretability of manual methods with the efficiency of deep
learning. This literature review not only informs our methodology but also positions our
work within the broader landscape of image processing.

i
The core of our approach lies in the meticulous implementation of a CNN. This choice is
driven by its proficiency in capturing spatial relationships within images, a crucial aspect in
colorizing photographs. The methodology extends to the careful curation of a diverse yet
contextually relevant dataset, forming the backbone of our model's learning process. The
training phase involves exposing the neural network to grayscale images alongside their
colored counterparts, enabling the model to extrapolate meaningful color information.
The journey is not without its challenges, as the implementation phase introduces intricacies
ranging from dataset curation nuances to the delicate trade-off between computational
efficiency and model complexity. However, these challenges provide valuable insights into
the complexities of the task at hand, enhancing the transparency of our approach.
In pushing the boundaries of image colorization, our project not only represents a
technological innovation but also prompts reflections on the ethical implications of digitally
altering historical imagery. As we delve into the results, discussions, and conclusions in
subsequent sections, this project serves as a testament to the transformative potential of
combining technology, art, and cultural preservation.

Introduction
In the vast tapestry of visual representation, black and white photography stands as a
timeless testament to moments captured in the absence of color. Its inherent simplicity and
ability to distill emotion and form have made it a revered medium in the narrative of human
history. However, as we journey through these monochromatic glimpses of the past, there
arises a profound curiosity – what if we could unveil the true colors of bygone eras? This
project embarks on a compelling exploration into the colorization of black and white images
through the application of deep learning techniques, seeking not only to add a spectrum of
hues but also to breathe renewed life into the visual stories of yesteryears.
The motivation behind this endeavor is rooted in the desire to bridge the perceptual gap
between historical photographs and the vibrant, lived experiences they encapsulate. While
black and white images evoke a sense of nostalgia and timelessness, they also run the risk
of distancing viewers from the dynamic reality they seek to represent. The intrinsic warmth
of sepia tones, the vividness of period-specific hues, and the nuanced play of light and
shadow – these elements are often lost in translation. Thus, our project is propelled by a
dual ambition: to pay homage to the aesthetic elegance of black and white photography
while concurrently enriching these visual narratives with the color palettes that defined their

ii
epochs.
The literature review unfurls a rich tapestry of methodologies and approaches in the realm
of image colorization. From early heuristic-based algorithms to the advent of convolutional
neural networks (CNNs), the evolution underscores a persistent quest to uncover efficient
and accurate means of infusing color into monochromatic canvases. Traditional methods,
while possessing a certain interpretability, often fall short in capturing the intricate details
and contextual nuances of diverse historical scenes. Contemporary deep learning
approaches, particularly CNNs, have demonstrated a remarkable capacity to address these
limitations, providing a data-driven avenue to learn and recreate the complex relationships
between grayscale and color.
As we navigate this dynamic landscape, our project situates itself at the crossroads of
tradition and innovation. The ambition is not merely to apply a superficial layer of color but
to engage in a nuanced reinterpretation of historical imagery. In doing so, we aim to foster a
more intimate connection between contemporary audiences and the narratives embedded in
these visual artifacts. This project, therefore, becomes a technological conduit for cultural
preservation, a means to reimagine and revitalize the collective memory encapsulated in the
black and white frames of the past.
This ambitious undertaking is characterized by its methodological rigor, employing a CNN
as the cornerstone of our colorization model. The chosen architecture, inspired by the
intricate interplay of neurons in the human visual system, is poised to decipher the complex
relationships within grayscale images and extrapolate meaningful color information. The
methodology further extends to the careful selection of a diverse dataset that encapsulates
the historical breadth and contextual relevance necessary for training a robust model.
In traversing this journey, we encounter not only the technological intricacies but also the
ethical dimensions of digitally altering historical imagery. The results and discussions that
follow will shed light on the successes, challenges, and potential future trajectories of this
project. Through the fusion of technology and art, this exploration into the colorization of
black and white images emerges as a testament to the ever-evolving dialogue between the
past and the present.

Literature Review and Research

COLORIZATION OF BLACK AND WHITE IMAGES: A SURVEY

Angelica Deigratia Pyngrope PG Student, Department of Master of Computer Applications

iii
 School of CS & IT JAIN (Deemed-to-be University)
Pawan Kumar Assistant Professor, Department of Master of Computer Applications School
of CS & IT JAIN (Deemed-to-be University)
In the ever-evolving landscape of image processing, the colorization of black and white
images has emerged as a captivating area of exploration, offering a bridge between the past
and present visual experiences. This comprehensive survey delves into the multifaceted
realm of colorization methodologies, synthesizing insights from a diverse array of research
endeavors to shed light on the evolution and current state of the field.
The motivation behind this survey is grounded in the recognition of the transformative
potential of colorization in enhancing the interpretability and emotional resonance of
historical photographs. As we traverse through the annals of literature, we witness the
evolution of techniques and approaches employed to infuse grayscale images with a
spectrum of hues. This journey serves as a foundation for understanding the nuances and
challenges inherent in the process of colorization.

Early endeavors in image colorization were often marked by heuristic-based algorithms,

where manual interventions played a pivotal role in determining the color palette. These
approaches, while demonstrating interpretability, faced limitations in scalability and
adaptability to diverse datasets. The survey meticulously analyzes these traditional methods,
providing valuable insights into their strengths and inherent constraints.
The advent of deep learning, particularly convolutional neural networks (CNNs), represents
a paradigm shift in the field of image colorization. As we navigate through the literature, we

iv
explore the successes and innovations brought forth by CNNs in capturing complex spatial
relationships within images. The survey delves into the architectures, training strategies,
and optimization techniques employed in leveraging the power of deep learning for more
automated and data-driven colorization processes.
Beyond the technical intricacies, the survey delves into the ethical considerations
surrounding the colorization of historical imagery. The act of digitally altering visual
artifacts raises questions about authenticity, cultural preservation, and responsible use of
technology. By examining the literature on these ethical dimensions, the survey aims to
contribute to the broader discourse on the societal impact of colorization techniques.
The survey also takes into account the diverse datasets used in colorization research,
acknowledging the importance of contextual relevance and historical accuracy. This
scrutiny enables a deeper understanding of how various methodologies adapt to different
genres of historical imagery, from portraits to landscapes.
As we traverse this comprehensive survey, the synthesis of historical perspectives,
traditional methods, and contemporary deep learning approaches offers a holistic
perspective on the colorization of black and white images. By distilling key insights from
the literature, this survey serves as a valuable resource for researchers, practitioners, and
enthusiasts, guiding them through the intricate tapestry of advancements and challenges in
this dynamic and evolving field.

Colorization of Black and White Images using

v
 Deep Learning

Pooja Aravind Palkar, Aparna Rajaram Patil, Jagruti Sahebrao Patil, Rachana Dhanawat
Ms, Ms, Ms, Dr
Department of Computer Science and Technology,
SNDT Women’s University, Mumbai, India

The colorization of black and white images, a captivating endeavor within the realm of
computer vision, has seen significant advancements facilitated by Convolutional Neural
Networks (CNNs). This literature review delves into key contributions by various
researchers, unveiling diverse methodologies leveraging CNNs for image colorization.
Richard Zhang's optimized solution utilized an extensive dataset and a single feed-forward
pass in a CNN. Focused primarily on the training phase, the approach achieved notable
results, fooling 32% of human subjects and showcasing the network's depth and complexity
in colorization tasks.

Domonkos Varga's exploration addressed the challenge of automated coloring for cartoon
images. The research recognized the unique characteristics of cartoon colorization, training
on a specific dataset curated for numerous cartoon images. However, the inherent
unpredictability in cartoon colors posed challenges requiring a personalized and moderate
approach.

vi
Shweta Salve's perspective involved using Google's image classifier, Inception ResNet V2,
in the colorization process. The system, divided into four parts, demonstrated satisfactory
results contingent upon adequate resources, serving as a proof of concept for integrating
powerful classifiers into colorization architectures.
Yu Chen proposed an approach tailored for the colorization of old Chinese movies.
Leveraging multiscale convolution kernels and combining features from the VGG-16
network, the model achieved effective colorization. The use of both existing and newly
created datasets underscored the adaptability of CNNs in handling diverse content.
V.K. Putri's method focused on transforming ordinary sketches into colorful images,
operating within the CIELab color space and employing a sketch inversion model. This
approach demonstrated versatility in handling hand-drawn sketches and various geometric
transformations, with commendable performance in uncontrolled situations.
In conclusion, these studies collectively contribute to the evolving landscape of colorization
using CNNs. Each approach reflects nuanced considerations, from extensive training
datasets to personalized methodologies for distinct image types. The review emphasizes the
interdisciplinary nature of this research, drawing from computer vision, neural network
architectures, and artistic considerations. As the field progresses, these diverse perspectives
enrich our understanding of the complexities and potentials in colorizing black and white
images.

Problem statement
In the vast repository of historical and monochromatic visual records, the absence of color
in black and white images often leaves a void in the representation of past moments. While
black and white photography holds an undeniable charm, the lack of vibrant hues can create
a disconnect between contemporary viewers and the lived experiences of the subjects
captured. The problem at the heart of this project lies in the inherent limitations of historical
visual narratives, hampered by the absence of color, which plays a crucial role in conveying
the richness and context of a scene.
The manual colorization of black and white images is a time-consuming and subjective
process, often reliant on the artistic interpretation of individuals. Traditional methods lack
scalability and consistency, making it challenging to apply a standardized color palette to
diverse historical photographs. Moreover, the sheer volume of images in historical archives
necessitates an automated and efficient solution to imbue these visual relics with authentic

vii
and contextually relevant colors.
Existing approaches to image colorization, including heuristic-based algorithms, face
challenges in adapting to the varied content, styles, and contexts present in historical
images. The need for a systematic and data-driven methodology is evident, and this is
where the potential of deep learning, particularly convolutional neural networks (CNNs),
comes into play. The complexity of historical scenes demands a model capable of
understanding spatial relationships and contextual nuances to generate accurate and visually
pleasing colorizations.
Furthermore, the application of colorization extends beyond the restoration of historical
images. It holds the potential to reshape the way we perceive and engage with visual
storytelling, offering a renewed perspective on the past. However, achieving this
transformation requires overcoming technical challenges such as network architecture
design, dataset curation, and addressing the ethical considerations surrounding the
modification of historical artifacts.
In light of these challenges, the problem statement encapsulates the imperative to develop
an advanced and scalable deep learning model specifically tailored for colorizing black and
white images. This model should not only automate the colorization process but also
preserve the authenticity of historical moments, fostering a deeper connection between
contemporary audiences and the visual narratives of the past. The successful resolution of
these challenges will not only contribute to the field of image processing but also redefine
the way we perceive and interact with our historical visual heritage.

Problem Solution

To address the intricate challenges posed by the colorization of black and white images in

viii
historical archives, an effective solution involves harnessing the power of deep learning,
with a specific focus on Convolutional Neural Networks (CNNs). The proposed solution
aims to create a robust and scalable model capable of automating the colorization process
while preserving the contextual authenticity of historical moments.

Dataset Curation:

 Begin by curating a diverse and extensive dataset of black and white historical
images. The dataset should encompass a wide range of genres, time periods, and
contexts, ensuring the model's exposure to the diverse visual characteristics inherent
in historical archives.
CNN Architecture Design:
 Develop a specialized CNN architecture tailored for the intricacies of historical
image colorization. The architecture should include layers capable of understanding
spatial relationships, contextual nuances, and intricate details within the images.
Consider using deep and wide networks to capture the complexity of historical
scenes effectively.
Training Process:
 Initiate a comprehensive training process using the curated dataset. Implement
techniques such as transfer learning, leveraging pre-trained models on large datasets,
to enhance the model's ability to generalize across diverse historical contexts. Fine-
tune the model to adapt to the specific challenges posed by historical images.
Context-Aware Colorization:
 Implement context-aware colorization mechanisms within the CNN architecture.
This involves incorporating features that consider the historical context, visual cues,
and artistic styles prevalent in different time periods. Context-awareness ensures that
the generated colorizations align with the authentic color palettes of the depicted
eras.
Ethical Considerations:
 Embed ethical considerations into the solution by incorporating features that allow
users to interact with and customize colorizations within acceptable bounds. This
ensures that the automated colorization process respects the integrity of historical
artifacts while offering a level of user engagement.
Validation and Testing:

ix
  Validate the effectiveness of the model through rigorous testing on diverse datasets,
including images not used during training. Implement quantitative metrics such as
PSNR and SSIM, as well as qualitative assessments involving human subjects, to
evaluate the model's accuracy and perceptual quality.
Scalability and Efficiency:
 Design the solution with scalability in mind, enabling the efficient colorization of
large volumes of historical images. This involves optimizing the model for
deployment on various computing architectures, ensuring accessibility and
applicability to extensive archives.
User Interface Integration:
 Develop a user-friendly interface that facilitates easy interaction with the
colorization model. The interface should allow users to explore and customize
colorizations, promoting engagement while maintaining a user's awareness of the
historical and ethical context.
 By combining the strengths of deep learning, contextual awareness, and ethical
considerations, this solution provides an effective framework for automating the
colorization of black and white historical images. The proposed approach seeks to
not only enhance the efficiency of the colorization process but also enrich the
viewer's experience by fostering a deeper connection with the diverse narratives
encapsulated in our visual heritage.

Methodology
Building a U-Net Architecture for Black and White Image Colorization

x
Dataset:
 Curation: Begin by curating a diverse and extensive dataset of historical black and
white images. This dataset should encompass a variety of genres, time periods, and
contextual scenarios, ensuring the model captures the diversity inherent in historical
archives.
 Preprocessing: Prepare the dataset by normalizing pixel values and resizing images
to a consistent resolution. Augment the dataset with techniques such as rotation,
flipping, and zooming to enhance the model's generalization capabilities.
Encoder:
 Feature Extraction: Design the encoder component of the U-Net architecture to
extract hierarchical features from the input grayscale images. Utilize convolutional
layers to capture both low and high-level features, allowing the model to
comprehend the spatial relationships within historical scenes effectively.
Decoder:
 Contextual Expansion: Develop the decoder segment to reconstruct the colorized
image from the extracted features. Employ transposed convolutional layers to
upsample and expand the spatial dimensions of the features. This facilitates the
recovery of intricate details and contextual information lost during the encoding
phase.
U-Net Architecture:

xi
  Symmetric Design: Implement the U-Net architecture with a symmetric structure,
featuring a contracting path (encoder) and an expansive path (decoder). Ensure skip
connections between corresponding encoder and decoder layers to enable the model
to capture fine details while maintaining global context.
Loss Functions:
 Perceptual Loss: Define a perceptual loss function that leverages pre-trained deep
neural networks, such as VGG-16, to measure the difference in features between the
colorized output and ground truth images. This encourages the model to generate
visually pleasing colorizations aligned with the perceptual characteristics of the
dataset.
 Pixel-wise Loss: Implement pixel-wise loss, such as Mean Squared Error (MSE), to
measure the pixel-level difference between the predicted colorized image and the
ground truth. This ensures accurate color reproduction and fidelity in the overall
image.
Training:
 Optimizer: Utilize an optimizer, such as Adam or RMSprop, to minimize the
defined loss functions during training. Adjust learning rates and exploration
strategies to fine-tune the model's convergence.
 Batch Training: Train the U-Net architecture in batches, ensuring efficient memory
utilization and faster convergence. Monitor the training progress using metrics such
as loss values and validation scores.
Testing:
 Validation Set: Evaluate the trained model on a separate validation set not used
during training. Use quantitative metrics, including PSNR and SSIM, to assess the
model's performance in replicating authentic colorizations.
 User Feedback: Incorporate user feedback during testing to gauge the perceptual
quality of the colorizations. This iterative process allows for refinements and
improvements based on human perception.

Conclusion
In conclusion, the project endeavors to bridge the perceptual gap between historical black
and white images and the vibrant realities they encapsulate. By adopting a U-Net

xii
architecture tailored for image colorization, the methodology presented here harnesses the
power of deep learning to create a model capable of automating this intricate process. The
careful curation of a diverse historical dataset ensures the model's exposure to the nuances
and contexts of different eras, fostering a comprehensive understanding of spatial
relationships and contextual intricacies.

The U-Net's symmetric design, integrating both encoder and decoder components with skip
connections, enables the model to capture fine details while preserving global context.
Leveraging perceptual and pixel-wise loss functions further refines the colorization process,
aligning the output with both human perception and authentic color palettes.
As the model undergoes training and testing, the iterative nature of user feedback ensures a
dynamic and user-centric approach. This project not only represents a technological
advancement in image processing but also serves as a testament to the harmonious
integration of artificial intelligence and historical preservation. The automated colorization
of black and white images becomes a transformative tool, reinvigorating our connection
with the past and offering a novel lens through which to explore the diverse narratives
embedded in our visual heritage.

Reference
1. COLORIZATION OF BLACK AND WHITE IMAGES: A SURVEY
(https://ijcrt.org/papers/IJCRT2203232.pdf)
2. Colorization of Black and White Images using Deep Learning
(https://www.jetir.org/papers/JETIR2205309.pdf)

xiii