Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

WiARL-UIT: A Public Dataset

for Simultaneous Human Activity Recognition and


Localization using WiFi Signals
Phat-Vo Le Thanh, Quyen Ng. Ph., Quy T. H.,
Minh Tuan Pham, Phuoc Nguyen T. H.*
(1) University of Information Technology, Ho Chi Minh City, Vietnam
(2) Vietnam National University Ho Chi Minh City, Vietnam.
{21522452, 21522533, 21522531, 19522479 }@gm.uit.edu.vn,
phuocnth@uit.edu.vn*

Abstract—WiFi-based human sensing technology has unique availability, resistance to light interference, overcoming terrain
properties such as widespread availability, resistance to light limitations, and respecting user privacy.
interference, overcoming terrain limitations, and respecting user For smart city applications, user activity and location aware-
privacy. It has been utilized in various fields. One interesting
application is simultaneous human activity recognition and lo- ness is a crucial factor. There are several interesting studies
calization, where the location and activity of a person standing on the potential for simultaneous human activity recognition
between WiFi transmitters can be predicted using received WiFi and localization using WiFi signals [18], [19]. These studies
signals and machine learning models. At the receiver, channel successfully demonstrate that it is possible to predict features
estimation process calculates the channel state information (CSI) of activities and features of locations within the same CSI sam-
which plays role as the input features for machine learning
models. However, CSI dataset collection is labor-intensive and ples. However, there are some limitations. In study [18], since
affected by hardware requirements and timing synchronization the dataset is not publicly available, it is challenging to conduct
between transmitters. Therefore, dataset is not publicly released additional research on the subject. The study [19] has publicly
in some studies. Other studies used a cable for timing between released the dataset with 1394 CSI samples in various activities
transmitters which limits the practicality in real-world scenarios. and locations. The WiFi transmitter and receiver are connected
In this study, we release the first public dataset with wireless
timing synchronization, namely WiARL-UIT, for simultaneous by a cable which provides perfect timing synchronization.
human activity recognition and localization. The dataset has 1500 However, the sync cable makes it impractical for real-world
CSI samples for 2 activities at 3 locations. For CSI dataset scenarios. Therefore, in this paper, we propose a framework
collection, we have developed a framework consisting of two for collecting WiFi CSI samples according to simultaneous
software-define radio (SDR) devices (USRP B200 Mini) trans- task of activity recognition and localization. The proposed
mitting IEEE 802.11n WiFi protocol. Finally, we have evaluated
the performance of various machine learning models in terms framework consists of software-defined radio USRP devices
of accuracy, precision, recall, and F1-score. As a result, the best with wireless timing synchronization. Using this framework,
accuracy is 97.0% with the CNN model, showing the feasibility of we have collected a dataset of 1500 CSI samples for various
the research direction of simultaneous human activity recognition locations and activities, then made it publicly available.
and localization. Dataset and source code are publicly share at The main contributions of our research can be summarized
https://github.com/F4tt/WiARL-UIT-Dataset.
as follows:
Index Terms—Activity Recognition, WiFi Human Sensing,
Indoor Localization, Machine Learning, CSI Fingerprints. • Developing a SDR-based framework with wireless timing
synchronization to obtain WiFi channel state information
for simultaneous human activity recognition and indoor
I. I NTRODUCTION
localization.
WiFi technology is now widely popular and virtually always • Collecting a dataset named WiARL-UIT with 1500 sam-
present in our lives. Transmitted WiFi signals are multi- ples for various locations and activities. The dataset is
path faded and can be affected by people standing on their publicly shared at https://github.com/F4tt/WiARL-UIT-
propagation paths. WiFi-based human sensing applications Dataset.
exploit the signal variations in amplitude and phase to predict • Evaluating the performance of various deep learning
the information contained in received signals. Currently, WiFi- models on the WiARL-UIT dataset in terms of accuracy,
based human sensing is extensively utilized in various fields, precision, recall, and F1-score. The highest achieved
including activity recognition [1]–[10], indoor localization accuracy is 97.0% with the CNN model, showing the
[11]–[14], and healthcare applications [15]–[17]. This success feasibility of the research direction. The source code is
is attributed to WiFi’s unique properties, such as widespread also shared by the link above.
were 88.13% for location recognition and 95.68% for action
recognition. This study demonstrates that it is possible to
separate features of actions and features of locations within
the same CSI sample.
To advance the research direction, we implemented an
SDR-based framework for collecting WiFi-based CSI dataset,
analyzed the dataset for various machine learning models, and
publicly shared the collected dataset.
C. Publicly Available Datasets
In recent years, several datasets have been publicly released
for research in human recognition using WiFi signals such
as [2], [9], [19], [20]. Because collecting CSI datasets is
Fig. 1. ”Application of simultaneous activity recognition and localization labor-intensive, these public datasets are valuable data sources
for smart-home scenarios.”
for researchers to evaluate ML models for human activity
recognition or indoor localization. However, for simultaneous
II. R ELATED W ORK human activity recognition and localization, the public dataset
in [19] is unique. In the study [19], the dataset was constructed
A. Human Sensing Using WiFi Signals with 6 actions performed sequentially at 16 different locations
Recently, many studies have been dedicated to the research in indoor environments with a total of 1394 data samples.
direction of human sensing using variations in WiFi signal What is special about this dataset is that CSI samples contain
waveforms [1]–[10]. Based on the channel state information information including both action and location, suitable for
received from channel estimation, many interesting ideas have models that can simultaneously recognize human activities and
been implemented to predict the human behaviors standing on indoor localization in the future.
the propagation path of the WiFi signals. For example, the To further contribute to the field of human sensing based
study [1] demonstrates the possibility of WiFi-based human on WiFi signals in general, and simultaneously recognize
recognition for various activities such as stumbling, slipping, human activities and indoor localization in particular, we have
fainting, sitting down, and standing up. This study highlights publicly released the WiARL-UIT dataset in this study.
the practicality of human action recognition in the medical
III. M ETHODOLOGY
field. Similarly, in [3], WiFi signals are used to detect human
hand gestures with 96% accuracy. This can be an interesting A. Recognition of Human Activity and Indoor Localization
idea for applying to smart city scenarios. Moreover, the studies Simultaneously using CSI
[3], [4] analyze the ability to recognize user keystrokes in the The initial foundation of this topic is based on wireless
conditions of small signal amplitudes. Finally, human local- sensing, specifically WiFi sensing. When it comes to WiFi
ization is also studied in [11]–[14] with impressive results. sensing, is defined as a sensing technology that detects phys-
ical or biological phenomena and converts that information
B. Simultaneous Human Activity Recognition and Indoor Lo- into electric signals that can be analyzed for various purposes.
calization Wi-Fi sensing uses Wi-Fi - also known as wireless local area
Although there have been a lot of studies on human recogni- networks (WLAN), to identify and interpret presence and
tion based on WiFi signals, there has not been much research motion. WiFi sensing detects and interprets motion through
on the joint task of activity recognition and localization two stages:
simultaneously. The Fig.1 demonstrates the application of • Pre-processing and initial filtering: The raw data provided
simultaneous activity recognition and localization for smart- by WiFi devices (referred to as ’CSI’ or ’channel state
home scenarios. information’) is filtered to eliminate pets and other non-
The first study on this topic is [18], which aims to precisely human movements.
identify actions taken despite location changes. A system • Higher-level processing: Artificial intelligence (AI) and
of eight wireless devices is deployed at two locations: in a machine learning algorithms then analyze the data for
laboratory and an apartment. All devices are synchronized advanced applications, such as interpreting breathing pat-
by wireless signals. The best results were achieved when terns.
recognizing the ”stand” action at different locations with an In the first stage, CSI (Channel State Information) can be
accuracy of 98% in the laboratory and 100% in the apartment. utilized for recognition tasks by analyzing the variations in the
The second study [19] has shown the feasibility of iden- wireless channel caused by the influence of the human body
tifying both actions and locations from Channel State Infor- between the transmitter and receiver devices. CSI is a signal
mation (CSI) samples. A 1D Convolutional Neural Network metric captured in an orthogonal frequency-division multiplex-
model was trained to distinguish the positions and actions of ing (OFDM) transmission system that describes the amplitude
volunteers with a pre-prepared dataset. The results achieved and phase changes across multiple subcarrier frequencies as
wireless signals travel from a transmitter to a receiver. The
OFDM system is modeled as:
y = Cx+n (1)
where x and y are complex vector indicating transmitted and
received signal respectively, n is a noise vector and finally,
C is a channel-information matrix. CSI is collected for each
subcarrier and the value for each of them is defined as a com-
(s)
plex number with a real component (Cr ) and an imaginary
(s)
component (Ci ). With that, denoting the subcarrier index as
s, we can calculate amplitude from raw CSI,
q
(s) 2 (s) 2
A(s) = (Ci ) + (Cr ) (2) Fig. 2. Broadcast and Receive systems
, and phase as well,
(s)
ϕs = atan2(Ci , Cr(s) ) (3)
In the next stage, we will use machine learning models
to train a dataset for the problem of simultaneous human
activity recognition and localization. The idea for simultaneous
recognition is described as follows: Given predefined labels
during the dataset collection process as shown in Fig.5, we
simultaneously differentiate information about the location (L)
and information about the action (A) recorded in the CSI
Fig. 3. System Framework
samples to determine which label the CSI sample belongs to.
Unlike traditional action recognition or location recognition
problems, here we require simultaneous accuracy in both at 3 locations, ”Right”, ”Mid”, and ”Left” within the WiFi
L and A. Suppose when comparing a sample CSI with a coverage area, described in Fig.4. This led to acquiring 6
predetermined label, the results show similarity in L, but labels, as depicted in Fig.5. For every label, we amassed
the result when comparing information about A returns ”not 250 CSI samples, totaling 1500 CSI samples for the dataset.
similar”, then we can conclude the sample CSI does not belong This dataset fulfills all the necessary criteria for our proposed
to that label. model’s objective of simultaneous recognition of both position
B. Proposed Platform for WiFi CSI Collection and action. As discussed in subsection II-C, there is a publicly
available dataset in [19] that aligns with the requirements
With two universal software radio peripherals (USRPs), we
for simultaneous recognition goals. This dataset offers the
implement the standard IEEE 802.11 protocol to collect CSI
advantage of creating numerous labels, up to 64 labels (6
samples. The USRP mainly consists of a motherboard (USRP
actions x 16 locations). However, the CSI samples per label are
B200-Mini) and a WiFi antenna (VERT-2450), which is used
limited (15 samples). To contribute to this field, we curated the
to broadcast or receive WiFi signals under the control of GNU
WiARL-UIT dataset with 6 labels but increased the number
Radio*. As shown in Fig.2, there is a view of the USRP and the
of CSI samples for each label to 250. The aim was to improve
details are listed below. Meanwhile, the assembling diagram
performance, reduce overfitting, increase accuracy, etc.
is shown in Fig. 3.
• USRP B200-Mini: A software-defined radio (SDR) de- B. CSI Fingerprint Analysis
vice with a wide frequency range from 70 MHz to 6 GHz We have simulated CSI fingerprint samples corresponding
and a user-programmable Xilinx Spartan-6 XC6SLX75 to different labels, as stated in Fig.5. We have presented 6
FPGA that can be embedded IEEE 802.11a/g/p protocol samples in Fig.6, each corresponding to a different label. The
to send and receive WiFi packages for CSI samples. x-axis in the figure represents the number of packets while the
• Antennas VERT-2450: To broadcast or receive WiFi y-axis represents the amplitude of the CSI. Each CSI sample
signals under the control of GNU Radio* consists of 52 subcarrier series is the number of orthogonal
• Computer and USB cables: To control B200-Mini when frequency division multiplexing (OFDM) [15] sub-carriers that
set in the same local area network as B200-Mini carry data in parallel in WiFi protocol. Fig.6 shows that CSI is
not too different despite the differences in location and action.
IV. DATA COLLECTION
In Fig.7 and Fig.8, we have provided examples of samples
A. Dataset that perform the same action in different locations. Addition-
To collect the WiARL-UIT dataset, we assigned a vol- ally, Fig.9 represents samples where we perform two different
unteer to execute 2 actions, ”HandUp” and ”ToTheLeft”, actions at the same location. Each sample with an x-axis
Fig. 6. CSI Fingerprints of 6 labels corresponding to the locations and actions
mentioned in Fig.5

Fig. 7. CSI fingerprints of the T oT heLef t” action in 3 corresponding


Fig. 4. Layout Diagrams with 3 locations LEF T , M ID and RIGHT location

value of 600. Based on Fig.6, the CSI are minimally affected


by changes in location and action, however, there is still a
significant change in the y-axis (amplitude).

V. M ODELS P ERFORMANCE
To obtain the results, we proceeded to use the collected
dataset WiARL-UIT to train machine learning models. Here
we propose using both complex neural network models such

Fig. 8. CSI fingerprints of the HandU p action in 3 corresponding location

Fig. 5. When a volunteer performs an action at a corresponding location, a


label is generated. The total number of labels is 6, which are A, B, C, D, E,
and F. Each label contains 250 samples.

Fig. 9. CSI fingerprints of 2 actions in the same location M ID


Fig. 10. Learning Curves and Losses when using CNN and MLP models
Fig. 11. Confusion matrix of CNN model
Models CNN MLP KNN SVM
Accuracy 97.0% 82.6% 80.6% 76.6%
Precision 97.0% 86.9% 81.1% 76.8% VI. C ONCLUSION
Recall 96.8% 80.3% 80.9% 76.6%
F1 96.9% 83.48% 80.6% 76.0% In this paper, we have developed a framework for WiFi CSI
TABLE I
M ODELS P ERFORMANCE .
dataset collection, which consists of two SDR devices using
wireless timing synchronization. Moreover, we have collected
and publicly shared the WiARL-UIT dataset - the first CSI
dataset with wireless timing synchronization for simultaneous
as CNN, and MLP, and simple machine learning models like human activity recognition and indoor localization. Among
SVM, and KNN. The results are listed in the Table.I evaluated models, the best performance of 97% accuracy was
obtained by the CNN model. The research has demonstrated
The relatively high results demonstrate the feasibility of the feasibility of signal recognition based on simultaneous
the simultaneous recognition problem. However, there is a differences in location and actions to differentiate between dif-
discrepancy in results between complex neural network models ferent signal labels. This opens up new avenues for combining
and simple machine learning models. With a large number of simultaneous recognition of multiple CSI sample information
data samples (1500) compared to the number of labels (6), the in the future. For future research, the ML model should
neural network models show superiority in the simultaneous be deployed on embedded devices and tested in real-world
recognition problem. Especially, the CNN model achieved scenarios.
the highest result of 97.0% in terms of Accuracy, Precision,
Recall, and F1 score after training for 100 epochs with a batch R EFERENCES
size of 128, learning curves in Fig.10 show that the CNN
[1] Sameera Palipana, David Rojas, Piyush Agrawal, and Dirk Pesch.
model performs well on both the training and test datasets, Falldefi: Ubiquitous fall detection using commodity wi-fi devices. Pro-
demonstrating good generalization ability and no overfitting ceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous
problems. Uniformity in the four parameters of the CNN Technologies, 1(4):1–25, 2018.
[2] Linlin Guo, Lei Wang, Chuang Lin, Jialin Liu, Bingxian Lu, Jian Fang,
model can be calculated using the confusion matrix in Fig.11. Zhonghao Liu, Zeyang Shan, Jingwen Yang, and Silu Guo. Wiar:
The corresponding results for each label are 100%, 98%, A public dataset for wifi-based activity recognition. IEEE Access,
91.4%, 98.2%, 97.6%, and 95.4% respectively. CNN followed 7:154935–154945, 2019.
[3] Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. Wigest:
by MLP shows significant differences in results with 82.6% A ubiquitous wifi-based gesture recognition system. In 2015 IEEE
accuracy for the MLP model. Meanwhile, simple machine conference on computer communications (INFOCOM), pages 1472–
learning models like KNN were trained in 2 seconds with 1480. IEEE, 2015.
an accuracy of 80.6%, and SVM was trained in 90 seconds [4] Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad.
Keystroke recognition using wifi signals. In Proceedings of the 21st
with an accuracy of 76.6%. The results of The SVM model’s annual international conference on mobile computing and networking,
results can be enhanced by tuning hyperparameters, optimizing pages 90–102, 2015.
margins, etc. However, this process is complex and demands [5] Yuxi Wang, Kaishun Wu, and Lionel M Ni. Wifall: Device-free
fall detection by wireless networks. IEEE Transactions on Mobile
significant computational resources, which may not be ideal Computing, 16(2):581–594, 2016.
when compared with employing Neural Network models. [6] Hong Li, Wei Yang, Jianxin Wang, Yang Xu, and Liusheng Huang.
Wifinger: Talk to your smart devices with finger-grained gesture. In
In summary, the CNN model demonstrates higher capability Proceedings of the 2016 ACM International Joint Conference on Per-
in learning and extracting features from CSI data, aligning with vasive and Ubiquitous Computing, pages 250–261, 2016.
the proposed model. Thus, we chose CNN as our main model [7] Hao Wang, Daqing Zhang, Yasha Wang, Junyi Ma, Yuxiang Wang, and
Shengjie Li. Rt-fall: A real-time and contactless fall detection system
for simultaneously recognizing human activities and indoor with commodity wifi devices. IEEE Transactions on Mobile Computing,
localization using CSI problem. 16(2):511–526, 2016.
[8] Isura Nirmal, Abdelwahed Khamis, Mahbub Hassan, Wen Hu, and
Xiaoqing Zhu. Deep learning for radio-based human sensing: Recent
advances and future directions. IEEE Communications Surveys &
Tutorials, 23(2):995–1019, 2021.
[9] Jieming Yang, Yanming Liu, Zhiying Liu, Yun Wu, Tianyang Li, and
Yuehua Yang. A framework for human activity recognition based on
wifi csi signal enhancement. International Journal of Antennas and
Propagation, 2021:1–18, 2021.
[10] Jian Liu, Hongbo Liu, Yingying Chen, Yan Wang, and Chen Wang.
Wireless sensing for human activity: A survey. IEEE Communications
Surveys & Tutorials, 22(3):1629–1645, 2019.
[11] Xuyu Wang, Lingjun Gao, Shiwen Mao, and Santosh Pandey. Csi-based
fingerprinting for indoor localization: A deep learning approach. IEEE
transactions on vehicular technology, 66(1):763–776, 2016.
[12] Ting Zhang and Yi Man. The enhancement of wifi fingerprint positioning
using convolutional neural network. In Proceedings of the 2018
International Conference on Computer, Communication and Network
Technology, 2018.
[13] Mai Ibrahim, Marwan Torki, and Mustafa ElNainay. Cnn based
indoor localization using rss time-series. In 2018 IEEE symposium
on computers and communications (ISCC), pages 01044–01049. IEEE,
2018.
[14] Xuyu Wang, Xiangyu Wang, and Shiwen Mao. Cifi: Deep convolutional
neural networks for indoor localization with 5 ghz wi-fi. In 2017 IEEE
International Conference on Communications (ICC), pages 1–6. IEEE,
2017.
[15] Biyi Fang, Nicholas D Lane, Mi Zhang, Aidan Boran, and Fahim
Kawsar. Bodyscan: Enabling radio-based sensing on wearable devices
for contactless activity and vital sign monitoring. In Proceedings of the
14th annual international conference on mobile systems, applications,
and services, pages 97–110, 2016.
[16] Hao Wang, Daqing Zhang, Junyi Ma, Yasha Wang, Yuxiang Wang,
Dan Wu, Tao Gu, and Bing Xie. Human respiration detection with
commodity wifi devices: Do user location and body orientation matter?
In Proceedings of the 2016 ACM international joint conference on
pervasive and ubiquitous computing, pages 25–36, 2016.
[17] Richard van Nee and Ramjee Prasad. OFDM for wireless multimedia
communications. Artech House, Inc., 2000.
[18] Jie Wang, Xiao Zhang, Qinhua Gao, Hao Yue, and Hongyu Wang.
Device-free wireless localization and activity recognition: A deep learn-
ing approach. IEEE Transactions on Vehicular Technology, 66(7):6258–
6267, 2016.
[19] Fei Wang, Jianwei Feng, Yinliang Zhao, Xiaobin Zhang, Shiyuan Zhang,
and Jinsong Han. Joint activity recognition and indoor localization with
wifi fingerprints. IEEE Access, 7:80058–80068, 2019.
[20] Jianfei Yang, Xinyan Chen, Han Zou, Chris Xiaoxuan Lu, Dazhuo Wang,
Sumei Sun, and Lihua Xie. Sensefi: A library and benchmark on deep-
learning-empowered wifi human sensing. Patterns, 4(3), 2023.

You might also like