Android Based Object Detection Application for Food Items
Hira Amjad, Maleeha Khan, Emaan Khan, Dr. Sadaf Abdul Rauf
Department of Software Engineering
Fatima Jinnah Women University
The Mall, Rawalpindi
Abstract: Many health-related problems emerge due to
improper diet and high quantity of calorie intake. Calorie
count cannot be recorded manually by people as it is not
convenient to record them before meal every time. To cope
up with this problem, we present a distinctive android
based food detection application which identifies the food
item from the given image and displays nutritional
information to the user. BMI calculator is also
implemented to provide ease in controlling weight along
with nutrients information. The proposed application uses
Convolutional Neural Network (CNN) which classifies the
image by learning image features. Two food datasets
named UEC-100 and ImageNet are analyzed by training
both of them to deeply examine the results of CNN in the
field of food image recognition and detection.
Keywords: Convolutional Neural Networks (CNN),
Machine Kernel Learning (MKL), Calorie Count, UEC-
100, ImageNet.
I. INTRODUCTION
There are many food categories in our daily routine
and differentiating among these categories based on
image is a challenging task due to the diversity of
food categories [ CITATION hoashi2010image
\l 1033 ]. Image recognition is a process in which
objects are detected or recognized in video or image
[CITATION Abadi2016 \l 1033 ]. It is a
simple process for humans but not an easy one for
computers. Humans can easily recognize a cup, an
apple, strawberry etc, but computer takes time to
learn images and face problems in recognizing
objects in images. A lot of work in machine
learning’s domain has been done in order to solve
these problems [CITATION Abadi2016 \l
1033 ].
Obesity and overweightness depends on body mass
index (BMI). BMI is a measure of body fat based on
height and weight. Overweight and obesity result in
more deaths than underweight[ CITATION
Poonsri2016 \l 1033 ], so weight should be kept
in control and for that it is important to know the
nutritional value of items that are being consumed.
For image recognition, there are various techniques
like bag-of-features (BoF), color histogram, Gabor
features and gradient histogram [CITATION
Abadi2016 \l 1033 ] which can help identify food
items.
Food is a vital feature for health and if not taken
properly, in terms of nutrients a high intake of
calories may cause serious health concerns.
[ CITATION Tammachat2014 \l 1033 ].
Automatic food recognition based on food images
assists people to grasp the knowledge of nutritional
values of their food [ CITATION Chen2016 \l
1033 ]. Food recognition is difficult, as food objects
are deformable unlike other objects like car, book,
pen etc and exhibit significant variations in
appearance [ CITATION yang2010food \l
1033 ]. For food recognition, techniques like
Convolution Neural Network, deep learning-based
food image recognition algorithms have particularly
proved useful.
Convolutional neural network is a method used for
visual recognition by computers[ CITATION
Yanai2015 \l 1033 ]. It is a kind of artificial
neural network that comprises of different neurons
where each neuron represents weights. CNNs have
played a very important role in image recognition and
classification [ CITATION Chen2016 \l
1033 ]. Image is given as an input and computer
reads it in binary form. Then it is stored in form of
array and passed through convolutional and fully
connected layers and then features of image are
learned using squares of input image. These provide
support to develop models by training them on
datasets and perform classification from these models
[ CITATION ciocca2016food \l 1033 ].
Image classification plays a vital role in providing
support to development of different object detection
applications[ CITATION Liu2016 \l 1033 ].
We present a food detection application which
identifies the food item from image using
convolutional neural network. After recognizing the
food item its nutrients information is displayed to
user on android interface. The application uses
TensorFlow for visual recognition. We worked on
Page 1|6
two data set and accuracy and loss values have been
calculated.
The Section II of paper describes literature review
and Section III describes analysis of UEC-100 and
ImageNet. In Section IV proposed methodology for
development of application is explain and Section V
concludes the paper and future work is presented in
Section VI. Table 4 discusses different methods for
image recognition and classification.
II. LITERATURE REVIEW
(Liu, et al., 2016) used CNN based deep learning
algorithm for identifying food categories. Food-101
dataset is used and results shows that Inception-Res
Net converges much faster and achieves top-1
accuracy of 72.55% and top-5 accuracy of 91.31.
Tensorflow is an open source library for machine
intelligence, (Abadi, et al., 2016). TensorFlow
operates at large scale and in heterogeneous
environments. It maps the nodes of a dataflow graph
across many machines in a cluster, and within a
machine across multiple computational devices,
including multi-core CPUs, general purpose GPUs,
and custom-designed ASICs known as Tensor
Processing Units (TPUs). This architecture gives
flexibility to the application developer whereas
previous “parameter server” designs the management
of shared state is built into the system, TensorFlow
enables developers to experiment with novel
optimizations and training algorithms.
For identifying food categories using image (Hoashi,
Joutou, & Yanai, 2010) has proposed the system that
automatically detects 85 categories of food. The
propose system is developed by combining Multiple
Kernel Learning (MKL) with characteristics of
different models such as color Histogram, Bag of
Features (BoF), Gradient Histogram and Gabor
feature. Food images are taken from mobile camera.
The results show that when only seventeen kinds of
images features are consolidated with MKL,
classification rate is improved by 62.52%.
Our heath is measured by what we eat and the food
content we take. (Ciocca, Napoletano, & Schettini,
2016), Proposed a new data set for diet monitoring,
this dataset evaluates different food recognition
algorithms. A dataset of 3616 images belonging on
73 food classes. The best performance has been
obtained by using Convolution-Neural-Networks-
based features. The dataset UNIMIB2015, as well as
the benchmark framework, are made available to the
research community.
For calculating nutrients using food images (Kawano
& Yanai, 2013) purpose region of food image is
segmented by GrubCut, then color histogram is
extracted and SURF based on bag-of-features is
applied. The result finding of this application shows
81.55% classification rate is achieved.
An automatic nutrition record method to read the
nutrient Guideline Daily Amounts (GDA) present on
packing materials proposed by (Poonsri,
Charoensiriwath, & Charoenpong, 2016). Accuracy
of 74.20%, light reflection and inclination cause the
result incorrect.
Object detection system is developed by combining
different features of classical tools from the domain
of deep learning and computer vision (Girshick,
Donahue, Darrell, & Malik, 2014).
III. PROPOSED METHOLOGY FOR
FOOD DETECTION APPLICATION
This section describes the proposed methodology for
android based object application. The aim of this
food Detection App is to provide ease to person in
accessing knowledge of daily food by just giving
image through mobile. This will help user to easily
check the nutrients and calories before lunch, dinner
or breakfast. The following methodology is proposed
to develop this application.
First, food items are detected using a pre-trained
model named VGG-16. This model is developed by
Visual Geometry Group in ImageNet ILSVRC-2014
and stood at first and second place in image
recognition and classification. It is 16-layer model
with weights of size 528MB with validation accuracy
7.5% and test accuracy 7.4% in top five classification
error on ILSVRC-2012. The model has been used
using Keras and implemented using Python language
on Ubuntu 16.0.
In second phase, database of nutrients has been made
in SQLite connected with android interface and it
comprises of different categories of food. Once food
item is detected by TensorFlow it will pass through
the android interface and nutrients of that food item is
Page 2|6
extracted and display to user. The application also analysis. Following eight categories are selected from
calculates body mass index by inserting height and both datasets to train them:
weight of a user. The purpose of BMI is to control  Burger
the weight along with checking nutrients of meal  Sandwich
taken. Both nutrients calculation and BMI plays very  Toast
important role in taking care of health.  Fried rice
 Steak
Android Based Application:  Pizza
 Egg roll and
Today the need of era is that we should have
 Rice.
enough information about how much healthy diet is
Now there are total 14 categories, seven categories of
our intake and what number of calories and nutrients’
each dataset. Every category of food contains 100
content are in there. But it is very difficult and time
food images in train directory and 10 images in
consuming for a person to manually records nutrients
validate/test directory.
and calories of food every time, he/she eats. Today
Classification of Train and Validate Directories:
technology has made it possible to automatically
detect food items and display information of nutrients
to user by making few swipes on phone.
The proposed android based food detection
application named “Smart Food” was implemented
keeping in mind the objective that by using it, people
can have health diet plan. The application provides
facility to a user to automatically detects food item by
just giving image to application and then displays
nutrients of that food item. Application uses the
concept of image recognition. It uses the method of
learning image features first and then detecting what
food item is. User can easily get percentage of
nutrients and then he/she can thinks of how healthy
the food is, is it beneficial to take or not, or how to
balance food quantity with health. Calories play a
major role in effecting ones’ weight. Keeping this
thing in mind, the user can also calculate Body Mass
Index to seek his/her weight information and manage Fig 1: Flow Diagram
diet accordingly. The diet plan can efficiently The results of application are shown below:
manage with the combination of BMI and nutrients
information. The application can also be used in
hospitals by patients in order to refrain from taking
food that is not suitable for their health.
Dataset:
UEC-100 and Image net images were used for
training purpose.
UEC-100
UEC-100 contains 100 kinds of food photos, where
each category of food includes 100 food images.
ImageNet (Food Synonyms sets)
ImageNet contains 1000,000 different synonyms sets
(synsets) organized in accordance with WordNet
hierarchy. Each synset in ImageNet contains average
1000 images. The images in this dataset represent
different categories such as animals, furniture, plant,
instrument, food etc. The paper mainly focuses on
food items, so eight synsets images are downloaded
by first registering to ImageNet repository.
After downloading, both datasets are observed and
common categories are chosen for first phase of

Page 3|6
 Fig 2: Food Detection
Fig 3: IP Address and Port Number
Fig 4: Upload Image
Fig 5: Classification Process
Fig 6: Nutrients Extraction
IV. ANALYSIS ON UEC-100 AND
IMAGENET DATA
Two datasets named UEC-100 and Image Net
have been taken to analyze them on the basis of
convolution neural network. Both datasets have been
trained from scratch using visual recognition library
called TensorFlow. Model for training have been
develop using high level API called Keras that uses
Tensorflow backend. Analysis has been done in
following three phases. In phase 1, the training and
validating data is from same dataset and same
categories of both datasets are chosen. The categories
of first phase are; rice, steak, fried rice, sandwich,
pizza, burger, eggroll and toast. For phase 2, train
and validate data in case of training of both datasets
are different. First UEC-100 is train on its own
images and validated with ImageNet images.
Similarly, ImageNet is trained on its own images and
validated on UEC-100 images. In phase 3, categories
of both datasets have been changed to check results
differently. UEC-100’s categories remain same but
for ImageNet, new categories have been chosen that
are; green salad, pasta salad, grains, bread, dairy,
fruit, vegetable and omelet. The following model has
been proposed for analysis based on training food
datasets.
Page 4|6

Fig. 7: Analysis Model
Results of first two phases of models of both datasets
are quiet similar but in phase II, loss values of model
become more negative in phase 2 with ImageNet
validation data, which means that UEC-100 has
achieved good validation when tested On ImageNet
categories. The results of accuracy and loss values of
all three phases are mentioned in Table 1, Table 2
and Table 3.
TABLE 1: LOSS AND ACCURACY RESULTS OF UEC-100 AND
IMAGENET (ANALYSIS PHASE I)
TABLE 2: LOSS AND ACCURACY RESULTS OF UEC-100 AND
IMAGENET VALIDATION
TABLE 3: LOSS AND ACCURACY RESULTS OF UEC-100 AND
IMAGENET
V. CONCLUSION
In this paper, first analysis of UEC-100 and
ImageNet has been discussed and then android based
food detection application is proposed. Analysis has
been done by training UEC-100 and ImageNet by
using CNN. The task of training has been
accomplished using machine intelligence library
called TensorFlow and image classification library
called Keras. Results of trained models shows that
UEC-100 dataset has shown good loss values when
validated with ImageNet images as loss values
become negative but accuracy values are are not
higher than 10% of all phases. The second
methodology is on development of food Detection
Android Application that detects food image given by
user using 16 layers neural network model named
VGG-16 and displays nutrients of detected food.
BMI helps a person to manage his/her weight and
BMI works best along with nutrients information of
food items. Results shows that Application has
successfully recognized different food items such as
strawberry, banana, cucumber and has displays
nutrients accordingly
VI. FUTURE WORK
In future, it is aim to use own model in application
detection module by first training it from scratch
using specific categories of food such as meat, rice,
fruits, vegetables, grains, fast food etc. The analysis
can also be done on more than two food datasets to
analyze the results of CCN for food recognition and
classification. This will give accurate results of
specific categories of food thus providing more
satisfaction related to health.
REFERENCES
[1] Hoashi, H., Joutou, T., & Yanai, K. (2010). Image recognition
of 85 food categories by feature fusion. Multimedia (ISM), 2010
IEEE International Symposium on, (pp. 296-301).
[2] Abadi, Ms., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J.,
others. (2016). TensorFlow: A system for large-scale machine
learning. Proceedings of the 12th USENIX Symposium on
Operating Systems Design and Implementation (OSDI). Savannah,
Georgia, USA.
[3] Bosch, M., Zhu, F., Khanna, N., Boushey, C. J., & Delp, E. J.
(2011). Combining global and local features for food identification
in dietary assessment. Image Processing (ICIP), 2011 18th IEEE
International Conference on, (pp. 1789-1792).
Page 5|6
[3] Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M. . . .
Zhang, Z. (2015). Mxnet: A flexible and efficient machine learning
library for heterogeneous distributed systems. arXiv preprint
arXiv:1512.01274.
[4] Chen, Y., Zhao, D., Lv, L., & Li, C. (2016). A visual attention
based convolutional neural network for image classification.
Intelligent Control and Automation (WCICA), 2016 12th World
Congress on, (pp. 764-769).
tuning. Multimedia \& Expo Workshops (ICMEW), 2015 IEEE
International Conference on, (pp. 1-6).
[15] Yang, S., Chen, M., Pomerleau, D., & Sukthankar, R. (2010).
Food recognition using statistics of pairwise local features.
Computer Vision and Pattern Recognition (CVPR), 2010 IEEE
Conference on, (pp. 2249-2256).A.R. S Bahai and B.R. Saltzberg,
Multi-Carrier Digital Communications Theory and Applications of
OFDM, Kluwer Academic Publishers, New York, 1-221, 2002.

[5] Ciocca, G., Napoletano, P., & Schettini, R. (2016). Food

recognition: a new dataset, experiments and results. IEEE Journal
of Biomedical and Health Informatics.

[6] Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich
feature hierarchies for accurate object detection and semantic
segmentation. Proceedings of the IEEE conference on computer
vision and pattern recognition, (pp. 580-587).

[8] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J.,
Girshick, R. Darrell, T. (2014). Caffe: Convolutional architecture
for fast feature embedding. Proceedings of the 22nd ACM
international conference on Multimedia, (pp. 675-678).

[9] Kawano, Y., & Yanai, K. (2013). Real-time mobile food

recognition system. Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition Workshops, (pp. 1-7).

[10] Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., & Ma, Y.
(2016). Deepfood: Deep learning-based food image recognition for
computer-aided dietary assessment. International Conference on
Smart Homes and Health Telematics, (pp. 37-48).

[11] Poonsri, A., Charoensiriwath, S., & Charoenpong, T. (2016).

The method to read nutrient quantity in guideline daily amounts
label by image processing. Knowledge and Smart Technology
(KST), 2016 8th International Conference on, (pp. 211-214).

[12] Pouladzadeh, P., Kuhad, P., Peddi, S. V., Yassine, A., &
Shirmohammadi, S. (2016). Food calorie measurement using deep
learning neural network. Instrumentation and Measurement
Technology Conference Proceedings (I2MTC), 2016 IEEE
International, (pp. 1-6).

[13] Tammachat, N., & Pantuwong, N. (2014). Calories analysis of

food intake using image recognition. Information Technology and
Electrical Engineering (ICITEE), 2014 6th International
Conference on, (pp. 1-4).

[14] Yanai, K., & Kawano, Y. (2015). Food image recognition

using deep convolutional network with pre-training and fine-

Page 6|6