2016 Fifth ICT International Student Project Conference (ICT-ISPC)

Wireless Door Access System with Face Verification

Pathompong Tungjitsirisun, Worayoot Pechsuwanrungsee, and Peerapon Siripongwutikorn
Department of Computer Engineering
King Mongkut's University of Technology Thonburi (KMUTT)
Bangkok, Thailand
p. tungjitsirisun@gmail.com, worayoot. p@gmail.com, peerapon.sir@kmutt. ac. th

Abstract-A traditional door access system using a keycard technology, face verification avoids physical contacts to users,
uses wired cables from many door units to the backend server for which is considered more hygienic.
authentication. The system installation is thus time consuming and This work proposes a wireless door access system with an
costly in a large floor area with many rooms. Besides, the
RFID keycard and face verification. The system is totally built
hardware and software in the system are proprietary to each
from off-the-shelf hardware components and open source
company offering the product. This prevents the user from
software. In order to bring solutions to the problems stated
extending the system or replacing door access units by using
components of another company. earlier, we explore the use of wireless technologies combined
with computer vision to improve operational flexibility and
In this work, we design and implement a wireless door access security measure of the door access system. Face verification
system where the door units communicate with the backend server prevents the fraudulent use of keycards while the keycard is
over a wireless network. To achieve better security and needed to identify a user to do face verification.
performance, the authentication is based on both a
This paper is organized as follows. Section II describes the
student/employee ID card and an image snapshot for face
design of the system architecture, where individual components
verification. The wireless implementation allows for much easier
and functions, the system operation, and the communication
installation and system extension. The system is built from custom
messages are explained. Section III provides the description of
off-the-shelf hardware components and open-source software,
which facilitate in-house implementation of the door units to the face verification method adopted in this work and how the
replace broken units or to add new units to the system. The system training is done. Section IV gives the details of system
has functioned successfully as designed. implementation based on the integration of the common off­
the-shelf hardware components and open source software
Keywords-Wireless door access, RFID, Face verification, libraries, and some test results. The conclusion is offered in
Python Section V.

I. INTRODUCTION II. SYSTEM ARCHITECTURE DESIGN

A today keycard system consists of a door unit and the

A. System Components and Operation
authentication server. The door unit, equipped with an RFID
As shown in Fig.I, the system contains two main parts­
reader, uses data stored in the user card or RFID tags for user
Door access unit (Front-End Device) and Backend server
authentication. Each RFID reader/node uses a wired
(Back-End Device). There is one or more door access units in
transmission media to communicate with the backend server,
the system and a single authentication server. B oth types of
typically located in a company server room. Such wiring leads
components are equipped with a network interface module that
to many issues, including installation difficulties, lack of
allow the door access units and the backend server to
operational flexibility, time and budget consumption.
communicate over a TCP/IP network.
Furthermore, the system implementation is proprietary, and it
is not possible to add or replace door units obtained from
The door access unit is installed at each room entrance
another company to the existing system. This means that the
and perform the following functions:
customer is stuck with this system. For example, when one of
o Control the magnetic door lock/unlock
the door unit fails, it could take very long to get the maintenance
o Receive data from RFID tags/cards of users.
service in case of poor after-sale services. Unsatisfied
customers would have to replace the whole system or install o Take face images of users

two different systems in parallel. o Transmit RFID data and images via the network to
Another issue of a typical door access system is that the the authentication server.
security can easily be violated. For instance, any person with a o Receive user information from the authentication
registered keycard can get pass the door even if he/she is not a server via the network.
card owner. This problem can be alleviated if the authentication
uses the face in addition to the keycard. Compared to fmgerprint

978-1-5090-1125-4/16/$31.00 ©2016 IEEE

 The backend server performs user authentication and
system management. In the authentication process, the backend
server does the following:
o Receive RFID data and user image from each door
unit
o Verify if the RFID data matches the captured user
image and instruct the door access unit to unlock the
door.
Access logs including user information, access time and
access status (success/failure) are kept in a database. The
management function allows for the administrator to add, edit,
and delete users in the system, and to generate access log reports
via the Web interface. The detailed system operation is depicted
as a flowchart shown in Fig. 2.
and timestamp is sent to the backend server for authentication
with the .JPEG image sent to the server via Secure Copy
protocol (SCP).
Response Message: After the backend server has made the
decision based on the information in the Request message, the
Response message is returned with the Node ID, Card ID,
Timestamp, and Response code. The following response codes
have been defined:
100: Accept with face and card verified
110: Accept with only card verified
210: Denied with face verification failure
220: Denied with card verification failure
230: Denied with user account expired

1----- - III. FACE VERIFICATION METHOD

- -- -eack-EOd-Oe�ic8j
i In this section, we describe the face verification

[ I
i technique together with a keycard to authenticate users. Unlike
,
Networ1<
Adapter !
! fmgerprint verification which requires a specific scanning
Hardware Modules
!

[ ,I[
hardware, face verification can easily be implemented by using
!

i
a simple USB Webcam to capture the user's face and then
Card Face
Authentication Verification verifying it with facial verification algorithm. In addition,

I hygiene matter when users have to share and use the fingerprint
Web-Based
Management
1 scanner with others.
!
, Many face verification algorithms nowaday exist to
I
Software Back-End Service , perform the task such as [3-5]. First, we detect the face of the
L_______________ J user by adopting the Haar-cascade Detection algorithm [1] due
to its widespread library implementation.
Fig. 1. System Architecture

Back-End

Receive
Authentication

l---t-I-- Data and

user's patrait
from Front-End

Authenticate
data received
from user's
tagcad

UNLOCK

USER

YES

Fig. 2. System Workflow Diagram

B. Communication Message
Facial Detection using Haar feature-based cascade
An IEEE 802.11 (Wi-Fi) network with AES encryption
classifiers is a machine learning based approach where a
is used to transfer data between the door access units and the
cascade classifier is trained with a lot of images with faces or
backend server. A simple client server protocol is sufficient for
positive examples and image without faces as negative images.
our system. From Fig. 1, two types of application messages are
After it is trained, it will then be used to detect faces in other
exchanged between each door access unit and the backend
images. A cascade classifier consists of multiple stage filters to
server over TCP - Request message and Response message.
detect the Haar-like features in the images.

Request Message: When a user taps his/her card to the door

access unit, the request message containing Node ID, Card ID,
 1t "
store

( (
Take user's Train model Store trained
,-_�I p ortra��mage PortraWimage I .� for prediction �eln .

'--___ �,--_O:_�_�_S:_J � 1
____

as dataset for Preprocessng from dataset

training Preprocessed (Training)

,-----......
J Images SER R

'----_../
-----+[ Face 10

Feature Vector
Location
USER

Fig. 3. Face Verification Training and Recognition Process

USB Interface

Raspberry Pi

SPllnlerface

Fig. 5. Hardware diagram of the door access unit

Fig. 4. Example training set.

In this work, we use pre-trained classifiers for face, eyes,

smile etc., which can be found in OpenCV Haar-cascade library
in the form of XML files such as "haarcascade­
frontalface default. xml".
The face verification implemented in this project are
written in Python using the combination of OpenCV library,
"Facerec" open library and other essential machine learning
libraries for mathematical calculations. After the user's faces Fig. 6. Door access unit prototype
has been detected, we use the face detected to compare the
facial features with the model of trained faces/collection of The backend server runs Ubuntu 14.04 LTS and are installed
faces in the system. Fig. 3 depicts the flowchart of how the with open source software libraries to perform each required
training for face verification is performed along with how the function:
face is recognized and Fig.4 shows an example of collection of • User database: MySQL 5. 6
user's faces to be used as training set. • Web management support: Apache 2. 4, php 6
The main output expected is the system's trained model • Face verification: Python 2. 7, OpenCV v.2.4. 11,
for recognizing a face must be able to predict user's incoming Haar-cascade pre-trained classifier, "Facerec" face
portrait from the door access unit with an acceptable level of recognition framework for Python [2], and Scikit­
accuracy and can verify that the person who taps the tag/card is learn Machine Learning library for python
the one he/she claims to be.

IV: SYSTEM IMPLEMENTATION AND RESULTS

This project aims to use open source libraries and off-the­

shelf hardware to provide the ease of use and to increase
flexibility for future upgrades, changes, and expansion. From
the system architecture described in Section II, we implement
the baseline door access unit using the following components
connected as shown in the diagrams in Fig.5:
Fig. 7. Test result of the face detection
• Raspberry PI model 2 B+ running Raspbian Linux to
control the peripheral devices. The code to perform face detection and verification is
• RFID module: MFRC522 connected to SPI written in Python based on the face cascade classifier library.
• Raspberry-PI-compatible USB Webcam: First, the USB Webcam is instructed to take a user snapshot,
Logitech nop USB Webcam C525 (V-U0023) with when the face is detected in the snapshot, the green box is drawn
python camera library provided. (minimum around the face as shown in Fig. 7. and passes it to the backend
resolution required: 480p) server.
• USB Wireless N Adapter: TP LINK TL-WNn5N
• Magnetic door control module connected to GPIO The python code at the backend server performs the face
interface verification by using the collection of preregistered users' faces
on the system's server and the face detected image captured
received from the door access unit. The feature extraction is
performed on the trained model and the image snapshot and the Fig. 9. Authentication time taken for one transaction.
feature vectors are compared to verify the user. We have tested
point is also used for another network application in the
on a few users and the system was able to correctly verify the
department that we have no control over. So, it is important the
door unit is placed closed to the access point as possible and the
access point should only be dedicated for the system.

V. CONCLUSION

From the implementation and study for this Wireless

Door Access system, we can see the advantages in the
application of wireless network technology to tackle the
limitations of traditional door access systems by rendering them
to be much easier to install or relocate because the wiring is
faces as shown in Fig. 8. eliminated. Along with the use of computer vision and machine
learning process to perform face verification, the system
Fig. S. Test results of face verification
security is enhanced to the level of using user's identity feature
The confusion matrix in Table 1 was obtained by using the in addition to an object like a keycard that bears no biological
training images of 6 users stored in the system to verify 20 relationship to users.
sample photos.
During the system implementation, we have
TABLE I. CONFUSION MATRIX OF FACE VERIFICATlON experienced lots of compatibility problems in integrating off­
the-shelf hardware components and the software. Software
Verified
support library services need to be revised and updated
11 8
-; continuously. We have also found that the wireless network
(TP: True Positive) (FP: False Positive)
.= also has its own problems such as the distance limitation,
Col
..r: 0 1
interference from other access points nearby, and the unreliable
(FN: False Negative) (TN: True Negative)
characteristics of the wireless channel that sometimes
a. Results from sample photos of 6 users in the system and I anonymous user frequently fail the communication between the door access unit
and the backend server. Therefore, the wireless footprint of the
The accuracy is calculated by using
door access units must be carefully planned in the real
deployment.
= True Positive+True Negative
AccuracY 0.6 60%
Number of sample photos
= =

Finally, for the face detection and face verification, the

It was found that most false positives are due to the mismatch model accuracy is a very important factor. The size of training
between the lighting conditions of the training images and the data set and the lighting conditions under which the sample
test images. photos are taken significantly affect the model accuracy.
Measuring the accuracy of the face recognition model is needed
The authentication time per transaction largely consists of the to fmd the model with the highest precision so that the system
image transfer delay between the door unit and the server over can operate effectively as possible.
the wireless network. We vary the horizontal distance from the
ACKNOWLEDGMENT
door unit to the wireless access point from and measure the
authentication time. The distance equal 0 means the door unit is We would like to thank Assoc.Prof. Thwnrongrat
right below the access point. The results are shown in Fig.9, Amornraksa and Mr. Thagorn Tangmankhong for participating
with each point being an average of five measurements. The as senior project committee to provide useful comments to
authentication time about 3 seconds is obtained when the door improve the quality of this work. We also thank our parents,
unit is placed right under the access point, and increases as the friends, seniors and juniors at the department of computer
door unit moves farther away from the access point because of engineering, KMUTT for all their support.
decreased bit rates. The surge at the horizontal distances 25m
and 30m are most likely due to other traffic because the access
REFERENCES

[I] Open Source Computer Vision. (2015). Face Detection using Haar
35.00 Cascades[online] Available at: http://docs.opencv. org/3.1.0/d7/
3: 30.00 Sb/tutorialJlY_face_detection.html#gsc. tab=O

.� 25.00
[2] Philipp Wagner. (2014). Facerec implements a face recognition
framework for Python. [online] Available at: http://bytefish.de/
.2 20.00 devIfacerec
"" [3] Sheng Zhang, Matthew Turk. (200S). Eigenfaces[online] Available at:
.S! 15.00 http://www.scholarpedia.org/article/Eigenfaces
;:
<l)
£ [4] Aleix Martinez (2011), Fisherfaces[online] Available at:
::> http://www.scholarpedia.org/article/Fisherfaces
..r:
[5] Harry Chao aMMAI (2010), Eigenfaces and Fisherfaces[online]
Available at: http://disp.ee.ntu.edu.tw/-pujols/Eigenfaces%20
o 10 20 30 40 50 and%20Fisherfaces. pdf
Horizontal distance from access point (m) [6] FIPS PUB 197. (2001). Advance Encryption Standard[online] Available
at: http://csrc.nist.gov/publications/fips/fipsI97/fips-197.pdf