SMARTPHONE’S

CAMERA
TECHNOL
OGY

pg. 1
2

Abstract

Favored by hardware development, since the mid-2000s, cameras can be

found in mobile phones. With the advent of the Apple iPhone they are
equipped with a multi-touch high-resolution display. Their included
battery and low costs make them attractive for smart camera applications.
This paper shows several scenarios, in which advantages and
disadvantages of smartphones are inspected. A real-life application is
given, which shows that a phone of this kind can be used for print
inspection and banknote authentication. Keywords: smart camera,
smartphones, banknotes, authentication

pg. 2
 ACKNOWLEDGEMENT

Karmaveer Bhaurao Patil College of engineering Satara, a place that Provided

me the many people, atmosphere to dream and experiences for Lifetime! The
work for this project was carried out at the Basic Sciences and Humanities
department, Karmaveer Bhaurao Patil College of engineering, Satara. The
making of this project involves help, encouragement, and Cooperation by many
people without whom, this project would not have been As, let’s say, a
“complete”. This piece of writing is dedicated to express my Sincere gratitude
to them.
I am grateful to Prof. (Dr.) A.C. Attar, Principal KBP college of engineering
Satara, Prof. Mr. A. B. Degaonkar, Dean Academics and Mrs. Sunita Ballal,
Head, Basic Sciences and Humanities department who provided a
motivating Environment in the department and supportive nature during
project work.
I would like to start my acknowledgments with my project guide,
Prof.A.B.Kadam for giving me time to time ideas and guidance related to project.
He set me on the track at the beginning and accelerated me when I go stuck
somewhere on my way.

I would like to thank my friends ……………………………..for their great

contribution in relieving the stress of work at the end of the day. They withstood
my absence in timely meetings, parties and chitchats. These acknowledgments
won’t be complete without mentioning heartfelt thanks and affection to my all
family members supported me and let me walk my way.
Some people contributed indirectly to this work. For those people, I might have
forgotten you, but you can have sure that this won’t be forever. Your contribution
has its own importance in my research career.

Thank you!!

pg. 3
4

INDEX

Sr. no Title Page no.

Introduction 1
1
2 Smart cameras-Definition and areas of application 1-3
3 Smartphones 4-5
4 Scenarios for smart cameras based on Smartphones 5-7
5 Increasing Classification Stability on Smartphones
6 Cameras in focus as smartphones image sensors hit
milestones
7 OCC Overview
8 Conclusions
9 References

pg. 4
1 Introduction

Progressing development in electronics and hardware results in more and more powerful, better,
cheaper, and smaller components that allow for new possibilities to employ smart cameras. In this
case, smart describes the fact that a digital image sensor is combined with a computing unit, which
delivers additional data about the content of a captured image via an additional interface. Hence, not
only is the environment recorded, but also a preprocessing takes place to understand the
environment. Enhanced performance of the hardware components leads to more and more complex
preprocessing steps and, consequently, to a deeper understanding of the surroundings.
Application of smart cameras takes place in user support, e.g. with measuring and inspection
tasks. They can help to reduce error margins caused by human beings that lose focus, get tired and so
on. Another area where these camera systems are employed is security. Cameras specialized on
security are able not only to record images, but also to detect motion and added or removed items.
Development of smart cameras started around the 1980s. In the mid-1990s, first commercial
cameras of this kind hit the markets. By today, sophisticated smart cameras can be found in many
areas. For example, since 2001, there exists a risen demand for surveillance technology (esp. in the
US), which led to according financial possibilities. Nevertheless, smart cameras are still rather
expensive, compared to cameras that only take pictures. Depending on requirements and equipment,
prices are in the five- to six-figure range (in Euro or US-$).
Favored by hardware development, since the mid-2000s, cameras can be found in mobile phones.
By 2012, these camera modules provide pixel resolutions that are comparable to or exceed those of
digital compact cameras. This raises the question of whether these cameras or the whole mobile
phone can complement or even replace intelligent cameras. This paper describes if and by how far
this is possible.
The following article is organized as follows. In the next section we describe in more detail the
definition of a smart cameras and their fields of application, while in Sect. 3 we describe smartphones
in detail. In Sect. 4 we discuss possible application scenarios for smartphone-based smart cameras,
and in Sect. 5 we show a real-life scenario and describe how it was actually implemented. Finally, we
summarize this paper in Sect. 6.

2 Smart Cameras – Definition and Areas of Application

In this section, smart cameras are defined, possible configurations are shown, and several application
scenarios are introduced.

pg. 1
2.1 Definition

Smart cameras were designed to mimic the human visual system. One goal was to develop an
intelligent optical sensor that detects and extracts relevant data from the enormous stream of image
sensor data, so that a subsequent application is enabled to make decisions based on these pieces of
abstract data. One of the most important properties of smart cameras is autonomous operation in
realtime
To reduce the data size, the smart camera has to decide which part of the data is relevant.
Consequently, a smart camera is defined as a stand-alone, integrated, and closed image processing
system. It features at least one image sensor, a computing unit, memory, and an interface, and uses
the intelligence first of all not to improve image quality.
The operational modular tool chain of a smart camera (cf. Fig. 1) starts with the light rays that are
directed by the optics to the image sensor. Modern sensors actuate some adjustment triggers
themselves, e.g. they adjust the aperture size so that the right amount of light is allowed to pass
through. Further, by applying lookup tables (LUT) they correct white point, color bias etc.

Processing Communication
Optic Image Sensor
Rays Unit Interface To/from user

Fig.1. Schematic representation of the structure of a typical smart camera [2].

The image sensor passes the data to a processing unit, the “brain” of the smart camera. This unit
can be a central processing unit like in a personal computer, a digital signal processor (DSP) or a
field-programmable gate array (FPGA). It computes complex image processing steps (high-level
image processing). In this step, pieces of information that are of interest are extracted from the image
data. The resulting pieces of information are passed to a human operator or a system (if the camera is
only part of a bigger machine) via a communication interface.

2.2 Available Configurations

The spectrum of modern smart cameras ranges from image processing systems with VGA resolution,
which are adapted to a problem at hand, over multifunctional high-performance devices that feature
image sensor resolutions of several megapixels and are capable to run highly complex algorithms in
realtime, to modular cameras (cf. Fig. 2). The latter may be connected, if necessary, to further
modules that expand the scope of features and adapt the camera to a certain task. If the job is
spatially distributed, some smart cameras are able to interconnect each other via wire or wireless
(distributed smart cameras, DSC). This way, they are able to solve problems together. The areas of
application for DSCs are manifold; often, they are employed for object tracking or surveillance

pg. 2
 Sensing Processing
Ethernet
Optic Memory
Serial
Image Sensor Digital signal
prozessor
(DSP ) WLAN

I/O
:
:
:

Fig.2. Architecture of a modular build smart camera [6].

Caused by advancements in technology, it is possible to integrate different hardware components

on one chip (system on a chip, SoC). This way, it is possible to implement image processing systems
in small format [6]. A further step towards miniaturization of components is done by integrating
preprocessing steps into the image sensor (smart sensor). Thereby image operations are done in
parallel to the output [7].
Depending on the task at hand, smart cameras are equipped with different image sensors and
lenses. Further, many manufacturers offer the possibility to exchange the lens. For example, some
cameras feature a c-mount lens thread [1].
Usually, smart camera systems offer a high-speed trigger input and can be equipped with a built-
in illumination. Further demand-oriented interface possibilities are analog or digital inputs/outputs
and wired or wireless network communication interfaces, for example to communicate with other
devices [1].

2.3 Areas of Application

There are many possible areas of application for smart cameras. Simple tasks demand attendance
control; more sophisticated tasks are measuring, testing, or application in robotics. Highly-
demanding tasks are e.g. distributed object tracking.
Since it is possible to build smart cameras in a compact way, even without moving parts, such as
fans or hard disk drives, these devices are normally not prone to disruption, compared to a standard
PC. Furthermore, when using a robust IP67 housing, they can be deployed in rough industrial
environments for continuous operation [1]. For a few years now, smart cameras are used in safety-
related areas such as automotive assistance.
The number of possible areas of application is much higher than the number where smart
cameras are in fact deployed. This is due to the still rather costly nature of these devices. Even though
the prices dropped steadily in recent years, caused by a higher integration and miniaturization, the
quantities are still rather small and, therefore, the prices remain on a high level. As a result, from an
economic point of view it is not always feasible to use smart camera systems. For example, when
manufacturing smallest series of a product, the quality check in the process depends on human
support. On the downside, this leads to the inclusion of human-based errors, e.g. special additional
test margins and larger error rates. Further, the operator has to be educated to be able to do his job.

pg. 3
3 Smartphones

In this section, we describe what a smartphone is and what it is capable to do, regarding the given
hardware. We focus on the camera module, since this would be the key when transforming a
smartphone into a smart camera.

3.1 Definition

A mobile phone is called a smartphone when it has the capability to be extended with small software
applications (apps) and when it offers a more advanced computing ability and enhanced connectivity
([5]). The increasing processor performance in recent years led to a hugely shifting usage behavior:
In the beginning, smartphones were used to e-mail or text each other in a more convenient way,
mainly by business users. Today, smartphones can run third-party apps, which extends the
functionality by far. The phone is not only a phone, but also a notebook, compact camera, gaming
companion, music player, internet surf station, satnav and so on.
The most important market players use essentially two different operating systems: Apple iOS
and Google Android. They share 85.6 % of all smartphones in the field, with Android being the
biggest player with a market share of 52.2 %.
3.2 Hardware Capabilities

Usually, smartphones are equipped with a large display. Since the advent of the Apple iPhone in
2007, large high-resolution multi-touch displays have become a de-facto standard. The finest
resolution is offered by Apple, with a 326-ppi display. The Samsung Galaxy Note N7000, one of the
largest smartphone currently on the market, has a screen size of 5.3 inch.
Furthermore, smartphones have a broad collection of sensors, e.g. gyroscope, accelerometer, GPS,
proximity or light.
Early smartphones used a single core processing unit with a clock rate of 600 MHz. By today,
multi-core processors (four to five cores) and clock rates of about 1.5 GHz are built in top-notch
models [3][4]. A Smartphone usually has two cameras that are described in the next subsection.

3.3 Cameras of Smartphones

Typical smartphones employ two different cameras: One at the screen side for video phone calls, and
one on the back. Normally, the first one has a resolution of about one Megapixel, the latter offers a
higher sensor resolution and is meant to be a replacement for a still or video camera. Since this is the
camera one would be interested in for application in a smart camera, we use the term camera
henceforth just for the high-resolution camera, and neglect the other one.
A typical smartphone camera has a resolution between five and twelve megapixels, with a trend to
bigger numbers. As with other compact cameras, this does not mean that the result improves.
Camera modules in smartphones lack a zoom lens (niche models like the Nokia Pureview 808
ignored at this point). These cameras have a sensor with a diagonal width of 4 to 7 mm, what makes
them prone to noise. The build-in illumination, often a LED- or Xenon-based flash, is only capable of
lightning objects near to the lens, e.g. portraits or close-ups.

pg. 4
 Big resolution figures equal a big memory demand. Today, this is why it is not possible to get raw
image data, which are important in image processing, from an Android phone in full resolution. The
result of an image capturing process is always a jpg-compressed picture. The compression factor
appears to decrease, though, meaning the data is less lossy compressed.

4 Scenarios for Smart Cameras based on Smartphones

After introducing smart cameras and smartphones, in this section we show different scenario
examples, where and how a smartphone can be deployed as a smart camera.

4.1 Deployment Possibilities for Smartphones as Smart Cameras

Due to the hardware specification and properties, smartphones can only be a restricted replacement
for a smart camera. Even though the processing unit is capable of delivering a sufficient performance
on complex image processing algorithms, the limited amount of communication interfaces and the
housing construction limits possible scenarios to mainly supporting tasks. It would not be realistic to
use a smartphone in its common appearance in a machine for measuring and test tasks. Industrial use
would be possible, though, when supporting a machine operator in his work. A consumer
smartphone or tablet pc is able to provide additional information about production steps, quality, and
so forth.

4.2 Operator Support

Modern production lines are able to produce a broad spectrum of products on one machine. A
permanent quality control makes sure that everything runs according to plans, but nevertheless,
random further quality checks inspect the production for possible machine errors. This step requires
detailed knowledge about the products. Therefore, employee training is necessary every time the
machine is changed to produce something else. Supporting smart cameras (smartphones) would be a
great help, since they would be able to present additional information about the new product on their
screen. Fig.3 juxtaposes a common smart camera and a smart camera based on a smartphone. Note
that the common smart camera has the need of an additional screen, and that it is stationary. The left
image shows an optical surface inspection system [12]. The right image shows a possible
implementation on a smart phone.
Another possible operation scenario is a chemistry lab. Different chemicals can be marked on
their containers with barcodes, which are read and interpreted by a smartphone. It is even thinkable
to interpret the substance appearance for identification. The advantage of smartphones compared to
traditional hand scanners is the larger display with higher resolution, which allows for showing more
and different pieces of information. Additionally, smartphones usually are smaller and lighter.

4.3 Print Inspection and Banknote Authentication

The examples above primarily describe the use of a smartphone as a smart camera not as a
replacement, but as a supporting tool integrated in a larger image processing system. Nonetheless,
there are problem-settings that can be solved with image processing algorithms on a smart phone.

pg. 5
One example that will be discussed in more detail is the use of a smartphone for forensic print
inspection.
Existing solutions use stationary devices, which provide a variety of features. Before it can be
used, an operator has to be thoroughly trained and expert knowledge has to be provided, for the
device as well as for the investigated objects.
As shown in [10], a smartphone is an apt device to authenticate banknotes with a smartphone.
The application used in this scenario is based on authentication algorithms described in [9][8]. It is
able to make an assumption about the print method based on the print structure. By use of the
Wavelet Transform,

Fig.3. Juxtaposition of a common smart camera (left) and one based on a smartphone (right). The smartphone
clearly is more mobile and more compact. It does not need additional peripheral equipment to find surface
errors and inform the user about it.

a banknote representation is separated into spectral components and analyzed afterwards.

Employing a new approach of a combination of Linear Discriminant Analysis (LDA) and Support
Vector Machines (SVM), the authenticity of the banknote is evaluated. The initial system based on an
established print inspection system, connected to a PC [8]. By transferring the authentication
algorithm to a smartphone, the possible application areas were stretched, and the inspection of prints
can now be done by untrained personnel. The different systems are depicted in Fig.4.

Implementing the authentication algorithms on a smartphone demands a new concept for certain
parts of some algorithms. Smartphone limitations that have an effect in this case are

– quality fluctuations of the camera module

– software limitations such as restricted or impeded access to raw image data

– constantly changing state of the surroundings, esp. light conditions, and – the position of the
banknote in respect to the smartphone.

pg. 6
In the next section we explain we compensated these obstacles.

Fig.4. Banknote Inspection Systems. The left system is based on a LabQMD quality measuring system and a PIAS-
II camera, the right one is based on a smartphone.

5 Increasing Classification Stability on Smartphones

After introducing the print inspection and banknote authentication scenario at the end of the last
section, in this section we show how we were able to increase stability on a smartphone and
therefore increase the reliability of the application.

5.1 Compensation of Quality Fluctuations

Camera modules in smartphones suffer from quality fluctuations during production. To reduce costs,
such modules put out an already optimized image; the operating system of the phone does not adjust
on any deviation. Quality deviations caused by the camera module manufacturer therefore lead to a
reduced image quality at the end. Diminished quality can show itself in a color cast, i.e.
the color channels are not properly adjusted, noise, bad focus, and so on.
These quality fluctuations can be compensated by considering them in the algorithms in form of
statistical distributions. By an appropriate choice of classification algorithms, measurement
deviations can be reduced to a minimum.

5.2 Strategy to Gain Stability

Use of a smartphone for real-time authentication of banknotes demands special procedures of

machine learning. Classification of objects should be as robust as possible, despite of unstable
capturing conditions. Further, the application should be reliable, despite of a limited number of
counterfeits for training. Falsepositive classifications have to be avoided.
Therefore, a training set is built or enhanced that considers possible variations that were not
possible with the industrial print inspection system. Additionally to the regions that are already part
of the training set, expanded regions from the edges are added (Fig.5)
When selecting an apt classification method, it has to be considered that the number of
counterfeits at hand is limited. On the other hand, the number of possible printing methods is limited
as well. Since false-positive classifications would question the whole application and lead to very
negative feedback, the

pg. 7
 Fig.5. To gain stability in classification, an existing training set is expanded by picture areas around the edges of
the original training set images. The picture shows the original region (solid lines) and the added regions
(dashed lines).

reliability of the classifier is most important. For this reason, the methods of machine learning that are
used in the authentication process have to be wellconsidered.
dMobile use results in unknown environment conditions when capturing. Several counter-
measures try to compensate this, e.g. shading correction and white balance adjustment. A shading
correction compensates inhomogeneous illumination. To be on the safe side, it has to be done
whenever the environmental illumination changes. A white balance adjustment, on the other hand,
corrects color casts by adjusting the color channels to a predefined reference value.

6. Cameras in focus as smartphones image sensors hit milestones

Smartphone imaging systems will continue

to see strong hardware upgrades this year, with Counterpoint Research’s latest Global Smartphone
Camera Demand-Side Report highlighting CMOS image sensor (CIS) content per smartphone will
expand to an average of 4.1. Despite the global components crunch, CIS growth is expected to grow by
double digits to reach almost 6bn units in 2021.

“A big driver has been triple-and-above main camera setups, which accounted for two-thirds of all
smartphones sold during the first half,” notes Tarun Pathak, Counterpoint’s director of smartphone
pg. 8
research. “What’s really interesting is where a lot of that growth is coming from – Africa, Latin
America, India and other emerging markets. As we move through post-COVID upgrade cycles,
especially in Android heavy markets, we’re seeing OEMs offer increasingly sophisticated camera
hardware to their customers across all segments.”

“High-resolution has also been an area of focus, with 48MP-plus becoming standard. Again, we’re
seeing emerging markets lead in growth; and 64MP is starting to become a major segment too. High-
res is very important for what is the most hotly contested price band globally – the wholesale $100-
$399 category. During the second quarter, two-thirds of devices were high-res and we expect further
share increases for the full year.”

A confluence of factors like more powerful chipsets, breakthroughs in AI image processing and
other hardware and software advancements are constantly delivering improved imaging experiences
for consumers in all segments. Features that were once only available on ultra-premium devices are
now emerging across OEMs’

broader portfolios – from contextual shooting, optical zoom and ultra-high res through to time-of-
flight and macro capabilities.

“Whether it’s a regional leader like TECNO catering to Nigerian youth, or Apple delivering DSLR
quality pics for SoCal influencers, imaging has become an even more core feature for all OEMs,”
observes Yang Wang, Senior Analyst for Africa. “And the bar being set in terms of the camera only
gets higher.”

“If you’re a product manager today delivering a quad cam device, then you’re probably thinking of
configuring wide + ultrawide + macro + depth. But the playing field changes quickly, and we’re likely
to see macro and ultrawide merge, leaving room for even more options like telephoto or time-of-
flight. Increasing choice and complexity is why algorithm development has become such a critical
factor in the success of camera systems,” states Ethan Qi, Counterpoint’s lead camera components
analyst.

“At the end of the day it’s not about camera or mega-pixel counts or how powerful your processor
is. It’s a combination of things:  How good is the integration? The AI algorithms? Is tuning tweaked
appropriately for the market?  It’s the sum of parts that delivers the experience,” according to Neil
Shah, Counterpoint’s Vice President of Research. “OEMs understand this, but it’s difficult to get right.
It’s as much art as it is science.”

pg. 9
 Background

Counterpoint Technology Market Research is a global research firm specializing in products in the
TMT (technology, media and telecom) industry. It services major technology and financial firms with
a mix of monthly reports, customized projects and detailed analyses of the mobile and technology
markets. Its key analysts are seasoned experts in the high-tech industry.

7.OCC Overview

A detailed and comprehensive study on the introduction to OCC technology is provided in Ref. . In
this section, we present a brief overview of the OCC system before discussing its implementation in
detail. The OCC system comprises LEDs and image sensors (cameras) which are used as transmitters
and receivers, respectively. Moreover, LEDs are used in almost every light-based application, such as
indoor illumination, street signals, and car tail lights and headlights . They quickly switch between
the ON and OFF states. By employing this property, LED light can be modulated using high
frequencies. figure shows the basic block diagram of an OCC system. Initially, optical channel
encoding is performed to detect and correct the error element while transmitting through the
channel with noise and interference induced by ambient light sources. Manchester coding (described
later) helps to eliminate the light-flickering effect. However, visual flickering may occur if the change
in the state of an LED is observed by the human eye (above 200 Hz is considered safe .Further,
encoded data is utilized to modulate the LED light at higher frequencies than a camera can detect.
Current camera image sensors usually show clear detection of frequency responses up to 6 kHz. All of
the tasks are performed by an LED driver circuit, which consists of a high-switching metal-oxide-
semiconductor field-effect transistor device, Arduino, and a power supply. Finally, LED light
transmits the modulated bits through the optical channel. The receiver extracts the information by
capturing and processing the frames with the help of an integrated OpenCV library as an image-
processing platform. In a rolling shutter camera, the LED images are captured as bundles of black and
white strips. Demodulation is performed on these images by measuring the widths and thresholding
of strips. After the demodulation and decoding tasks are completed, the output data is shown on a
display.

pg. 10
 8.Conclusion

In this paper we showed that it is possible to use a smartphone as smart camera, under certain
restrictions. The nature of the housing of a smartphone as well as the fact that a display is integrated
on the back of the camera limit the deployment scenarios. Nonetheless, a smartphone has its
advantages over industrial standard smart cameras. It has a built-in battery, which makes it
autonomous; it provides a high-resolution multi-touch screen, which is a very versatile output device
as well as a highly flexible input device.
We showed several scenarios such as a supporting smart camera device in production or a
chemistry lab, and inspected the use-case of a banknote authentication device more in-depth. In this
context we pointed out some intrinsic obstacles one has to overcome when converting an existing
application to a smartphone.
Overall, it has to be emphasized that the hardware of smartphones just recently reached a certain
level at which it can be used as a smart camera. As technology advances further, the possibilities to
use smartphones in ways they were not intended to be used will increase further.

REFERENCES
1.wikipedia.org
2.www.bbc.com
3.www.slrlounge.com
4.gizmogyan.com
5.www.androidauthority.com

pg. 11
THANK YOU.

pg. 12