2018 International Conference on Signal Processing and Information Security (ICSPIS)
A Business Card Reader Application for iOS devices
based on Tesseract
Bello Ahmed Dangiwa
School of Mathematical and Computer Science.
Heriot Watt University.
bd2@hw.ac.uk
Abstract—As the accessibility of high-resolution smartphone
camera has increased and an improved computational speed, it is
now convenient to build Business Card Readers on mobile phones.
The project aims to design and develop a Business Card Reader
(BCR) Application for iOS devices, using an open-source OCR
Engine – Tesseract. The system accuracy was tested and evaluated
using a dataset of 55 digital business cards obtained from an online
repository. The accuracy result of the system was up to 74% in
terms of both text recognition and data detection. A comparative
analysis was carried out against a commercial business card
reader application and our application performed vastly
reasonable.
Keywords—Optical Character Recognition, iOS, Image
Processing, Tesseract OCR, Natural Language Processing
I. INTRODUCTION
Business cards are widely used on a daily basis by
professionals. In professional meetings/gatherings, business
cards are exchanged as formal greetings and as a way of building
networking. The number of business cards will excessively
grow, which makes managing or communicating to card owners
an issue. The best way to manage and ensure communication is
by digitizing it. Numerous researchers proposed and built
systems using flatbed card scanners to manage and digitalize the
business cards. A downside to this approach is the lack of
portability of flatbed scanners. As the accessibility of high-
resolution cameras, low price and better computation speed in
smartphones no longer possess concerns and also with the
significant growth in usage of mobile phones. It becomes more
convenient to use smartphones for business card digitalization.
The project largely used iOS frameworks and libraries to
achieve a fully functioning application. The project targets to
provide a BCR application that will resolve challenges faced
with business cards usage by providing a digitized solution. The
application offers users the ability to scan business cards, extract
details and save to phone contacts. It is built for iOS devices in
XCode v9.2 with Swift Programming Language.
II. LITERATURE REVIEW
L. Xi-Ping et al. [1] presents the design and execution of a
business card reader application using a built in mobile phone
camera. Another technique is proposed that centers on multi-
resolution analysis of document images, with an aim to solve the
challenge faced in restricted resources of mobile devices. The
proposed technique will aid in improving computational speed
and lessens memory usage in the image processing stage by
spotting the text regions in downscaled images and then
analyzing spotted regions in the original image.
Authorized licensed use limited to: GITAM University. Downloaded on February 29,2024 at 12:35:01 UTC from IEEE Xplore. Restrictions apply.
978-1-7281-0257-3/18/$31.00 ©2018 IEEE
Smitha S Kumar
School of Mathematical and Computer Science.
Heriot Watt University.
Smitha.kumar@hw.ac.uk
The proposed method starts off with a business card image
captured using a mobile phone camera. In the image processing
section characters, words and lines are spotted from the image.
Then, the optical character recognition unit is used to recognize
the identified characters. Finally, characters and texts are
classified and important information from the business card are
extracted. Such as name, address, phone number, etc.
To assess the application, experiments were carried out using
a database of 400 images from 100 various business cards. The
average time taken to recognize a single image was
approximately 2.5 seconds in a 200MHz ARM CPU. The
recognition rate of symbols and single characters was
approximately 98.4%.
In conclusion, the proposed system still faces challenges on
segmentation of characters in blurred images. Detecting text
regions in complex background still remains a challenge. Time
taken to recognize one image should be considered poor.
R. Jana et al. [2] explores another method of effective optical
character recognition based on texture and topological features
of text image. For improved performance, texture and
topological features of text image such as corner points, features
of different regions and character area to convex area ratio are
calculated. Character verification is carried out by comparing the
extracted character with the template of all character for a
similarity measure check.
Two datasets are considered, the training dataset and the test
dataset. Preprocessing and feature extraction is carried out in
both cases. Extracted features from test data are compared with
training data features to get desired outputs.
In preprocessing stage, the text image gets converted into a
binary image to be processed. It is justified to be easier to work
with pixel values 0 and 1 of the binary image. Binary 1 (one)
embodies the letters and binary 0 (zero) embodies the
background. At this instant, individual text lines gets separated
from the binary image by calculating sum of all values in a row.
When sum is zero, new text line is recognized and separation is
done. After all text lines have been extracted, individual letters
are to be extracted. Letters are extracted by calculating the sum
of all values in a column. When sum is 0, individual characters
are identified and separated. In this manner, all individual
characters get separated (including digits, characters and
punctuations).
In the Feature extraction stage, Harris Corner method is
used. The character image corner points are calculated. Then the
total area of extracted character image gets calculated with the
exact number of pixels in character image. Using number of
pixels, the convex area of the character is computed. The result
is the ratio of convex area to total area, and all individual
characters are extracted.
 2018 International Conference on Signal Processing and Information Security (ICSPIS)
Result = Convex Area / Total Area
The paper uses three fonts (Arial, Lucida Fax and Berlin
Sans) as the training data set. Using the training data set features
for extracted character recognition. The experimental results for
fonts in the training data set are; Arial = 87.14% accuracy, Berlin
Sans = 94.28%, Lucida Fax = 64.28%, Cambria = 14.28% and
Times New Roman = 21.42% (Results for a case using 70
characters).
The accuracy for fonts not in the training dataset appears to
be very poor. To achieve higher accuracy, fonts have to be used
in the training dataset. This makes the proposed method not very
scalable.
Reference [2] uses fonts as a training data set for character
recognition which is not an effective method to be implemented
in a mobile phone due to memory limitations. It also [2]
indicates the use of texture and topological feature such as
corner points of the image to enhance performance in the
extraction stage. To compare [2] to [1], both papers are
seemingly using Optical Character Recognition for character
extraction. However, it can be argued that [1] will be more
effective for mobile phone implementation as it requires lesser
memory in the image processing stage.
[3] and [4] suggests the use of different methods for reading
business cards. [3] propose the use of augmented reality in
business card readers. The business cards images are used as
image markers in order to retrieve business card information
from a cloud database. [4] proposes the use of Multi-Color
Composite (MMCC) barcode. The barcode is used to store the
digital version of the business card, which can be retrieve by
scanning the barcode. The above two proposed methods are
debatably not the best of approaches for business card readers
due to their lack of flexibility and their limitations. Hence, this
project will focus on using Optical Character Recognition for the
Business Card Reader to be developed.
[5] use a similar approach to [2] in using the four corner
points of the business cards in extraction stage for better
performance. Conversely, [2] appears to have provided a more
detailed information on processes taken to achieve its extraction.
The four corner points technique will be looked into for possible
implementation in this project.
The proposed method in [6] presents a new noise and picture
region filtering technique – using the Connected Component
(CC).
[7] proposes to develop an android application that allows
mobile phones to capture text-based image, extract text from the
image, translate it into English and finally speech it out. The
application is developed with the Tesseract OCR Engine, Bing
translator and smartphones’ built-in speech out technology.
As seen in [8], the binarization process used may be better
than the other techniques used in other papers. A neural network
was used in character classification, the method also improved
the classification accuracy. After the above review, an image
processing module shall be added to the system to enhance
accuracy of the OCR engine.
III. METHODOLOGY
Implementation of system functionalities were completed
following the Agile – Scrum Methodology. System
requirements and functionalities were accomplished in sprints –
Authorized licensed use limited to: GITAM University. Downloaded on February 29,2024 at 12:35:01 UTC from IEEE Xplore. Restrictions apply.
iterative increments. After each sprint completion, the
functionality was tested and required improvements were made.
The Development of the application followed the object-
oriented programming principles with classes built to be easily
extended or reused.
A. Technologies used
1) UI Kit Framework
UI Kit framework provides the architecture for
implementing interfaces, event handling, etc. Interaction
between user, system and application is also managed by this
framework. The development started by connecting and
defining all the components in the UI to the View Controller
classes for implementation. These includes the flash button,
guide button, labels, the camera capture button, text fields, etc.
2) AV Foundation Framework
AV foundation is a framework that allows configuration of
built-in iPhone cameras for capturing, recording
and processing of photos, videos and audio. We explicitly setup
this framework to capture photos (scan
business cards) and additional support for rear flash camera
usage only.
3) Core Image Framework
Core Image is an image processing framework offered by
Apple that delivers high performance for
processing still images and videos. It provides numerous built-
in features for image enhancement and
processing. This framework is used as the image pre-processing
module in the application.
4) Tesseract OCR Engine – Text Recognition
Ray Smith [10] from Google Inc. provided an overview on
the open-source Tesseract OCR Engine. He focused on the
unique aspects of the OCR engine, these includes line finding,
the classification methods, and an adaptive classifier.
A step-by-step pipeline process followed in Tesseract OCR
Engine (Fig. 1). The first step is the connected component
analysis, it inspects nested outlines with the number of
child/grandchildren outlines and stores outlines of the
components. This makes it easy to detect inverse text and detect
white texts on black background. Stored outlines are grouped
into blobs. Blobs are ordered into text lines, and then regions are
examined for proportional texts. Text lines are further broken
into words based on character spacing.
The second step of the process starts by attempting to
recognize single words in turns. Acceptable recognized words
are passed to the adaptive classifier to be used as training data.
The Adaptive classifier uses the accepted words to improve
accuracy in recognizing words in the lower part of the page. As
the adaptive classifier might have learned and improved in
recognizing words on the page, a second pass is run again to
improve recognition of words on the near top of the page. The
final step to remove fuzzy spaces is taken.
Fig. 1. Tesseract OCR Architecture (Simplified)
 2018 International Conference on Signal Processing and Information Security (ICSPIS)
5) Natural Language Processing API
Natural Language Processing API uses machine learning to
understand texts with features like language identification,
named entity recognition, lemmatization etc. The API was used
for processing recognized text from business cards.
6) Contacts framework
Contacts Framework is used to access user’s contacts. We
used the framework to add new contact information into the
user’s contacts.
IV. IMPLEMETATION
This section covers the detailed processes as well as
decisions taken to complete the development of the proposed
application. The application is developed in XCode IDE v9.2
with Apple’s Programming Language – Swift 4.
The main camera screen was completed first as it is the first
functional requirement of the system. This functionality allows
the user to scan a business card, the scanned card will be passed
to the next controller (screen) to be processed.
Capture session (Fig. 2) is the most essential component of
AV Foundation for capturing photos or videos. In AV
Foundation, a CaptureSession object is used to handle sessions
and also manage the flow of data in physical devices – as inputs.
The capture session was setup as photo preset in order to acquire
the highest resolution quality photo (in output). Using Discovery
Session, the device was configured to use the back-facing
camera with built-in Wide-Angle Camera settings. A codec type
of jpeg used to maintain a high resolution with low memory
usage.
After successfully setting up camera for capture sessions. A
capture button is set to use the AVCapturePhotoSettings object.
When a picture is taken, the output is passed to the
AVCapturePhotoOutput method to be further processed down
the application. The diagram below depicts the flow and
relations in the components:
Devices
Input
Capture Session
Output
Fig. 2. Relations between Components of AV Foundation (Camera Usage)
A. Image-Preprocessing Module
After receiving the captured image from the view controller
(camera), it is passed to the Core Image Framework to be
processed and enhanced. The first step taken in the
preprocessing module is to detect rectangular features in the
image. Using the CIRectangleFeature object, the image is
scanned for rectangular areas and information about identified
regions is returned. Once the features are acquired, perspective
correction is applied, this uses the identified rectangular features
to add CGPoint on the corners of the detected rectangle (Bottom
Right, Bottom Left, Top Left and Top Right). CGPoint
structures store X and Y value properties. The image orientation
is then set and scaled. The final phase is cropping the image to
gain only the rectangular card.
Authorized licensed use limited to: GITAM University. Downloaded on February 29,2024 at 12:35:01 UTC from IEEE Xplore. Restrictions apply.
B. Text Recognition – Tesseract OCR Engine
The pre-processed (Final) image is then passed into the OCR
Engine – Tesseract for text recognition. Tesseract OCR is added
into the project as a framework. The framework uses the English
trained datasets provided by Tesseract for text recognition.
Using the tessractCubeCombined engine mode as it delivers the
best accuracy in Tesseract. The passed image is further
processed using the in-built Tesseract image processing module.
The image is segmented and binarized (black and white), the
binarized image is then handed for text recognition. The
recognized text is stored in NSString which will be used in the
next phase of the application. NSString is a plain-text Unicode
object that supports comparing, searching and modification of
strings.
C. Natural Language Processing
After Tesseract OCR recognizes the text, first and last names
needed to be recognized since Natural Language Processing API
didn’t have support for this. A separate method was created for
checking first and last names. The recognized text is passed to
this method. A character set was initialized containing all the
English alphabets (both lower and upper case). The set of
characters was used to eliminate any other unwanted characters
such as numbers or signs as it not commonly-found in names.
Afterwards, lines containing two words separated by a single
space were checked. The reason behind this, is that a lot of
business cards contain only the first and last name in this manner
as the aim is for a larger set of business cards to be fully
recognized. Finally, white spaces are trimmed and retrieved
names are added to their respective text fields. To check for
email address, Regular Expressions were used to identify email
strings from the recognized text and adds the email to its text
field.
Then the Natural Language Processing API was used to
identify phone number, link and address. This uses the
NSDataDetector class to match the recognized text with
predefined data patterns for phone number, links and address.
Finally, all the identified entities are automatically populated to
their respective text fields for the user.
D. Adding to Contact
The contacts framework is used in this function to save
contact information into phone contacts. The business card
information filled in the text fields are automatically added into
the phone contacts. It is connected to a UI button which on click
triggers the CNSaveRequest and stores the new contact.
E. Search for Social Network accounts
This was implemented by connecting each social network
search function to a UI button. User can search for full name of
the card owner in Facebook, LinkedIn and Twitter by tapping
the corresponding button. It uses an NSURL class object to
access the URL of the social networks, with full name used as
a search query.
The diagram below (Fig. 3), depicts the stages and modules
of the system from scanning a business card to successfully
saving retrieved contact details. These stages collectively make
up the whole system.
 2018 International Conference on Signal Processing and Information Security (ICSPIS)

Fig. 3. System Architecture (Simplified)

Fig. 4. AETHER BCR (This Project) (Comparative Analysis)

Figure 5 displays the accuracy of each retrieved element for

V. EVALUATION ABBY BCR (Commercial Application).
The system was tested and evaluated for efficiency, efficacy
and performance. The system was evaluated in two different
phases; theoretical and experimental assessments. The tests and
evaluations conducted are; User Interface and Usability, System
Accuracy Testing and a comparative analysis with a commercial
BCR application.
Overall, the usability feedback suggests the application is
very easy to use, all functionalities worked accurately, and users
were satisfied. The system accuracy was tested and evaluated
using a dataset of 55 digital business cards obtained from an
online repository [9]. The accuracy result of the system was up
to 74% in terms of text recognition and data detection combined.
However, this application is built for best use with physical Fig. 5. ABBY BCR (Commercial Application) (Comparative Analysis)
business cards. And the fact that, quality of some of the pictures
used from the dataset appeared to be poor.
In a comparative analysis with a commercial BCR
REFERENCES
application. We tested both applications in the same office
lighting conditions against 15 physical business cards. We [1] L. Xi-Ping, L. Jun, and Z. Li-Xin, "Design and implementation of a card
reader based on build-in camera,” vol. 1, ed. USA, 2004, pp. 417-420.
checked the accuracy of each data retrieval that is; name, phone,
[2] R. Jana, A. R. Chowdhury, and M. Islam, "Optical Character Recognition
email, address and URL. The table (Table 1) below shows the from Text Image," International Journal of Computer Applications
results (in %) acquired from each of the data. Technology and Research, vol. 3, no. 4, pp. 240-244, 2014.
[3] V. Hing and H. K. Khoo, "Business card reader with augmented reality
engine integration,” vol. 398, ed, 2017, pp. 219-227.
Data This Project ABBY BCR [4] S. K. Ong, D. Chai, and A. Rassau, "A robust mobile business card reader
using MMCC barcode," ed, 2011, pp. 656-661.
Name 53.8% 61.5% [5] G. Hua, Z. Liu, Z. Zhang, and Y. Wu, "Automatic Business Card Scanning
with a Camera," ed, 2006, pp. 373-376.
Phone 100% 92.3% [6] A. F. Mollah, S. Basu, N. Das, R. Sarkar, M. Nasipuri, and M. Kundu,
"Text Region Extraction from Business Card Images for Mobile Devices,"
Email 83.3% 100% 2010.
Address 9% 81.8% [7] S. Ramiah, T. Y. Liong, and M. Jayabalan, "Detecting text based image
with optical character recognition for English translation and speech using
URL 87.5% 75% Android," ed, 2015, pp. 272-277.
[8] K. S. Bae, K. K. Kim, Y. G. Chung, and W. P. Yu, "Character recognition
Overall Accuracy 66.52% 81.92% system for cellular phone with camera,” vol. 1, ed. USA, 2005, pp. 539-
544.
Table 1. Comparative Analysis Result [9] Purl.stanford.edu. (2011). Dataset: Stanford Mobile Visual Search
Dataset. [online] Available at: https://purl.stanford.edu/rb470rw0983
This project got an overall accuracy of 66.52% and ABBY [Accessed 4 Apr. 2018].
BCR application got a higher overall accuracy of 81.92%. Our [10] Smith R. “An overview of the Tesseract OCR engine”, InDocument
application performed highly reasonable against a commercial Analysis and Recognition, 2007, (ICDAR 2007), Ninth International
application. Conference on 2007 Sep 23, Vol. 2, pp. 629-633.

Figure 4 below shows the accuracy (in %) of each retrieved

element (name, email, phone, etc.) for this project.

Authorized licensed use limited to: GITAM University. Downloaded on February 29,2024 at 12:35:01 UTC from IEEE Xplore. Restrictions apply.