Professional Documents
Culture Documents
Smartphone Camera Based On Visible Light Communication
Smartphone Camera Based On Visible Light Communication
Abstract—The paper proposes a novel camera-based receiver One of the key candidates to solve the so-called capacity
for visible light communications for a short range mobile-to-mobile crunch in certain applications is the visible light communica-
communications link. The receiver captures data from the screen of tions (VLC), which potentially offers 10,000 times more band-
a transmitting smartphone and uses the speeded up robust features
algorithm to effectively detect it. The receiver performs a projec- width capacity than the RF based technologies [7]. The VLC
tive transformation to accurately eliminate perspective distortions technology is mostly based around the intensity modulation of
caused by the displacement of the devices. The paper also intro- white light emitting diodes (LEDs) (in-organic [8] and organic
duces a quantization process in order to suppress the inter-symbol [9]), which can be switched on and off at a very high rate [10],
interference resulting from the dynamic nature of the environment. thus enabling data communications, illuminations, sensing as
A range of experiments are carried out in order to evaluate the
system performance when the position parameters are varied. We well as indoor localization [11]. LEDs are widely used in every-
show that the proposed system is capable of achieving a very high day infrastructures including homes, offices, street and traffic
success rate of 98% in recovering the transmitted images under lights and smartphones [12]. In display devices such as smart-
test conditions. phones or computers, the individual elements of the pixel arrays
Index Terms—Image processing, Projective transformation, can be independently modulated [9] and captured with a camera
Smartphone, speeded up robust features (SURF), visible light (i.e., optical camera communications) in order to recover the
communications (VLC). transmitted information [13].
Current smartphones have such capabilities in detecting infor-
I. INTRODUCTION mation. For example, in [14] a CMOS camera sensor was used
to capture the data from visible LED light source. A similar
ADIO frequency (RF) technologies are extensively used
R in wireless based communications systems. RF applica-
tions vary from short range indoor communications such as near
approach was proposed in [15] where a message was encoded
using Red, Green and Blue (RGB) LEDs in order to improve
the data rate. In [16], an experimental screen-to-camera VLC
field communications, Wi-Fi or Bluetooth to mobile cellular system was reported, where a 256-LED transmitter and a com-
networks [1] up to medium-to-long range links such as millime- mercial webcam were used as the transmitter (Tx) and receiver
ter wave backhauling [2] and satellite-to-Earth communications (Rx), respectively. In order to detect LED transmission on the
[3], respectively. The explosion in mobile devices usage over captured frames, four LEDs were used as a reference. To lo-
the last decade has led to the ever increasing usage of the lim- cate the reference LEDs, an edge detection scheme based on
ited and expensive radio spectrum. For instance, wireless and Hough transform was used in [17]. In [18], the capacity of
mobile devices will exceed traffic from wired devices by 2016 smartphone camera (SC) communications considering the in-
(54%) [4]. In Europe, the total frequency spectrum for public herent distortions and varying the distance between the Tx and
mobile communications is now over 500 MHz [5], with the the Rx was investigated. However, in this work the interference
additional spectrum becoming available in the future at higher- between pixels was not considered in the analysis. In [19], a
frequency carriers. The spectrum has become rather a limited camera based VLC link was demonstrated within an aircraft
resource when considering the rapid growth in the number of cabin, where successive 2D visual codes were displayed on the
interconnected mobile users and the emergence of Internet of in-flight entertainment screen and captured by SC.
things paradigm. RF based technologies are facing a substantial In this paper we investigate, for the first time, the perfor-
demand for the capacity that simply cannot be met with current mance of the SC link under different mobility conditions. This
spectral allocation [6], therefore new solutions must be found. is a timely and important investigation as it takes advantage of
the widely used screens, available in public places and personal
Manuscript received December 28, 2015; revised March 22, 2016 and May devices. The system is portable as it can be readily implemented
11, 2016; accepted June 6, 2016. Date of publication July 18, 2016; date of on smartphones, without any hardware modifications or addi-
current version August 12, 2016.
R. Boubezari, H. Le Minh, and Z. Ghassemlooy are with Optical Com- tions. Any type of digital data can be converted into transferable
munications Research Group, Northumbria University, Newcastle upon Tyne images consisting of thousands of bits per image, which can
NE1 8ST, U.K. (e-mail: rayana.boubezari@northumbria.ac.uk; hoa.le-minh@ then be displayed on the screen of a smartphone and captured
northumbria.ac.uk; z.ghassemlooy@northumbria.ac.uk).
A. Bouridane is with Computer and Electronic Security Systems, Faculty of by the camera of a second smartphone to recover the informa-
Engineering and Environment, Northumbria University, Newcastle upon Tyne tion within a single frame. Similar to a dynamic 2D barcode,
NE1 8ST, U.K. (e-mail: ahmed.bouridane@northumbria.ac.uk). it offers the possibility of transferring a significant amount of
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. data by simply displaying it on the screen of a smartphone
Digital Object Identifier 10.1109/JLT.2016.2590880 screen. The potential of such a technology is even higher if RGB
0733-8724 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
BOUBEZARI et al.: SMARTPHONE CAMERA BASED VISIBLE LIGHT COMMUNICATION 4121
Fig. 7. (a) Transmitted image, and (b) perspective distortions on the received
image.
TABLE I TABLE II
COMPARISON OF SVLC WITH EXISTING SYSTEMS EXPERIMENT PARAMETERS
Transmitter 32 inch 5 inch 21.5 inc LCD 4.8 inch Number of cells (M × N ) 3750(75 × 50)
LCD screen smartphone screen smartphone Number of pixels per cell 10 × 10
screen screen Number of cell colour states 2 (black and white)
Receiver Off-the smartphone Tablet smartphone 1280 × 720
shelf camera camera camera Tx smartphone screen resolution (4.8 inches HD super
camera (Samsung S3) AMOLED)
Cells Black/ Coloured Black/white Black/white Rx smartphone front camera (Samsung 1.9 MP
white S3) effective pixel
Distance Up to 14 m 5 inch 5m Up to 1.2 m Rx smartphone back camera (Samsung 8 MP
Number of 81x81 800 x 400 15 x 15 Up to S3) effective pixel
cells 170x120 Reference camera 18 MP
(Sony QX-10) effective pixels
Link length d 10 to 110 cm
Parallel rotation angle θ 0 ◦ to 360 ◦
Tilting angle α 0 ◦ to 75 ◦
570 lux
Measured ambient level (lux) (Note: ISO office standard
(Normal office lighting) lighting level is ∼400 lux)
Fig. 10. Success rate of data recovery against the link distance d for hard
thresholding (HT) and soft thresholding (ST) schemes. Fig. 11. The successful rate of data recovery as a function of the rotation
angle θ.
results only within the range of 0◦ to 20◦ . These results again [4] Cisco, “Cisco Visual Networking Index: Forecast and Methodology, 2015–
indicate the importance of camera quality on the system per- 2020, White Paper” ed, 2015.
[5] D. Lewin, P. Marks, and S. Nicoletti, “Valuing the use of spectrum in the
formance. The results also show that the use of Otsu method EU,” GSMA, London, UK, Jun. 2013.
is more advantageous than the HT method. It is to be noted [6] W. Yuanquan and C. Nan, “A high-speed bi-directional visible light
that new generation of Smartphones will offer higher resolu- communication system based on RGB-LED,” China Commun., vol. 11,
pp. 40–44, 2014.
tion cameras and screens, thus offering higher potential when [7] D. A. Basnayaka and H. Haas, “Hybrid RF and VLC systems: Improving
adopted in SVLC. user data rate performance of VLC systems,” in Proc. IEEE 81st Conf.
Next we carried out experiments by changing the number of Veh. Technol., 2015, pp. 1–5.
[8] M. Hoa Le et al., “100-Mb/s NRZ visible light communications using a
cells displayed on the Tx’s screen in order to extensively evaluate postequalized white LED,” IEEE Photon. Technol. Lett., vol. 21, no. 5,
the performance of the Rx capability under different threshold pp. 1063–1065, Aug. 2009.
schemes. The number of cells was varied from 3,000 to 100,000 [9] P. A. Haigh et al., “Wavelength-multiplexed polymer LEDs: Towards 55
mb/s organic visible light communications,” IEEE J. Sel. Areas Commun.,
over a 15 cm link distance. For Sony camera and Samsung Back vol. 33, no. 9, pp. 1819–1828, Sep. 2015.
camera, the performance decreases slightly when increasing the [10] D. O’Brien, L. Zeng, L.-M. Hoa, G. Faulkner, J. W. Walewski, and
number of transmitted cells until it drops after 90,000 cells. S. Randel, “Visible light communications: Challenges and possibilities,”
in Proc. IEEE 19th Int. Symp. Pers., Indoor Mobile Radio Commun., 2008,
However, the Samsung front camera is more affected whilst pp. 1–5.
increasing the number of cells, and the recovery rate drops [11] H. Le Minh et al., “High-speed visible light communications using
below 70% when transmitting over 30,000 cells. multiple-resonant equalization,” IEEE Photon. Technol. Lett., vol. 20,
no. 14, pp. 1243–1245, Jul. 2008.
Note that 3750 black/white cells were transmitted [12] Y. Wang, N. Chi, Y. Wang, L. Tao, and J. Shi, “Network architecture
per frame (i.e., 3750 bits). Given that a data rate of a high-speed visible light communication local area network,” IEEE
R= (FPS − 1)×M×N×log2 (K), where FPS is the camera rate Photon. Technol. Lett., vol. 27, no. 2, pp. 197–200, Jan. 2015.
[13] L. Pengfei et al., “Experimental demonstration of RGB LED-based op-
and one frame per second would be devoted for the detection tical camera communications,” IEEE Photon. J., vol. 7, no. 5, pp. 1–12,
frame. K is the number of colours used in the data encoding. Oct. 2015.
A data rate of 112.5kbit/s could be achieved with real time [14] C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using
a CMOS camera sensor for visible light communication,” in Proc. IEEE
processing. However, R might be slower due to the need for Globecom Workshops, 2012, pp. 1244–1248.
synchronisation at the Rx. Using 10,000 cells and maintaining [15] Casio unveils prototype of visible light communication system using
an 80% transmission rate, a three times higher R could be ob- smartphones at CES - 2012 - News - CASIO (2012) [Online]. Availabe:
World.casio.com
tained with RGB images, thus offering ∼300 kbit/s. It can be [16] N. Trang, L. Nam Tuan, and J. Yeong Min, “Practical design of screen-
anticipated that R would be much higher with the new high to-camera based optical camera communication,” in Proc. Int. Conf. Inf.
quality camera phones. In addition, grayscale or RGB methods Netw., 2015, pp. 369–374.
[17] F. Dai, N. Zhang, and J. Xue, “Primal sketch of images based on empirical
could be adopted instead of the black/white scheme. mode decomposition and Hough transform,” in Proc. 3rd IEEE Int. Conf.
Ind. Electron. Appl., 2008, pp. 2521–2524.
[18] A. Ashok, S. Jain, M. Gruteser, N. Mandayam, Y. Wenjia, and K. Dana,
VII. CONCLUSION AND FUTURE WORKS “Capacity of pervasive camera based communication under perspective
distortions,” in Proc. IEEE Int. Conf. Pervasive Comput. Commun., 2014,
This paper demonstrated a mobile VLC system by means pp. 112–120.
of investigating transmission and detection of encoded images. [19] T. Fath, F. Schubert, and H. Haas, “Wireless data transmission using visual
SURF, projective transformation and quantization were appro- codes,” Photon. Res., vol. 2, pp. 150–160, Oct. 2014.
[20] S. V. Kothiya and K. B. Mistree, “A review on real time object tracking in
priately used to realise objective detection. Tests were carried video sequences,” in Proc. Int. Conf. Electr. Electron. Signals Commun.
out under various position-displacement conditions for smart- Optimization., 2015, pp. 1–4.
phone VLC. The proposed system offered the suppression of [21] C. Schmid and R. Mohr, “Local grayvalue invariants for image retrieval,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 5, pp. 530–535,
path loss, inter-symbol interference and perspective distortions. May 1997.
The achieved results outlined a reasonable performance for stan- [22] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient
dard smartphones and provided a benchmark for short range data alternative to SIFT or SURF,” in Proc. IEEE Int. Conf. Comput. Vis., 2011,
pp. 2564–2571.
communications as evaluated against the Rx camera type. [23] B. Su, S. Lu, and C. L. Tan, “Robust document image binarization
In future the experimental work will be further extended to technique for degraded document images,” IEEE Trans. Image Process.,
different mobile platforms. Different testing conditions (outdoor vol. 22, no. 4, pp. 1408–1417, Apr. 2013.
[24] L. Dongju and Y. Jian, “Otsu method and K-means,” in Proc. 9th Int. Conf.
or sunlight), hand-shaking and fast movements will be investi- Hybrid Intell. Syst., 2009, pp. 344–349.
gated for a more extensive evaluation of SVLC. [25] N. Otsu, “A threshold selection method from gray-level histograms,” IEEE
Trans. Syst. Man Cybern., vol. 9, no. 1, pp. 62–66, Jan. 1979.
[26] S. D. Perli, N. Ahmed, and D. Katabi, “PixNet: Interference-free wireless
REFERENCES links using LCD-camera pairs,” presented at the Proc. 6th Annu. Int. Conf.
Mobile Comput. Netw., Chicago, Illinois, USA, 2010.
[1] A. H. Sakr and E. Hossain, “Analysis of K-tier uplink cellular networks [27] T. Hao, R. Zhou, and G. Xing, “COBRA: Color barcode streaming for
with ambient RF energy harvesting,” IEEE J. Sel. Areas Commun., vol. 33, smartphone systems,” presented at the Proc. 10th Int. Conf. Mobile Syst.,
no. 10, pp. 2226–2238, Oct. 2015. Appl., Serv., Low Wood Bay, Lake District, U.K., 2012.
[2] C. Dehos, J. L. Gonzalez, A. De Domenico, D. Ktenas, and L. Dussopt,
“Millimeter-wave access and backhauling: The solution to the exponen-
tial data traffic increase in 5G mobile communications systems?,” IEEE
Commun. Mag., vol. 52, no. 9, pp. 88–95, Sep. 2014.
[3] T. Hirano, K. Miyazaki, M. Hatamoto, R. Yasumitsu, K. Hama, and
F. Watanabe, “RF-MEMS for reconfigurablesatellite antenna,” in Proc.
2nd Eur. Conf. Antennas Propag., 2007, pp. 1–5. Authors’ photographs and biographies not available at the time of publication.