Gait Assessment using the Kinect RGB-D Sensor*
Jingbo Zhao, Frank E. Bunn, Jacob M. Perron, Edward Shen, Robert S. Allison

Abstract— Patients with concussions, strokes and
neuromuscular disease such as Parkinson's disease, often have
difficulties in keeping balance and suffer from abnormal gaits.
Gait assessment conducted by a physician or therapist in clinics
is standard clinical practice for assessing such injuries.
However, this approach is subjective, leading to potential
problems of unrepeatability, poor sensitivity and unreliability.
To conduct the assessment in an objective way, a
computer-based gait assessment system is designed and
presented in this paper. The system performs assessments on
dynamic balance and gaits by analyzing the skeleton frames of a
subject captured by the Microsoft Kinect RGB-D sensor.
Results show that the proposed system effectively scores
subjects.
I. INTRODUCTION
Patients with concussions, including athletes, may have
difficulty in keeping balance and these difficulties are often
reflected in changes in gait. These problems have to be
assessed before proper therapies can be given or before
determination of suitability for play or other activities can be
made.
Current methods for assessing human gaits are subjective
and are usually conducted by a physician at a clinic or a
trainer on the field. Since they are performed in a subjective
way, the results may differ between assessors and may vary
even for the same evaluator at different times [1]. While the
standard tests have been shown to have moderately good
inter-rater and test-retest reliability, there is considerable
room for improvement [2][3][4]. Moreover, reliability is
lower with inexperienced assessors [4] but trainers who do
such assessments relatively infrequently must make many
locker room or on-field decisions. Computer-based
assessment systems offer the potential for the process to be
conducted in an objective and more repeatable way.
To realize the aim, it is desirable to develop a computer
vision system for gait assessment. The Microsoft Kinect
sensor has received great attention for its promising
applications in balance assessment. Previous studies have
*Research supported by the NSERC Engage program and the NSERC
CREATE Training Program in Vision Science and Applications.
Jingbo Zhao is with Department of Electrical Engineering and Computer
Science, York University, Toronto, Canada (e-mail: jingbo@cse.yorku.ca).
Frank E. Bunn is with PH.D. Associates Inc., Toronto, Canada (e-mail:
bunn@phdassociatesinc.com).
Jacob M. Perron is with is with School of Computing Science, Simon
Fraser University, British Columbia, Canada (e-mail: jperron@sfu.ca).
Edward Shen is with Bubl Technology Inc., Toronto, Canada (e-mail:
ed@bublcam.com).
Robert S. Allison is with Department of Electrical Engineering and
Computer Science, York University, Toronto, Canada (corresponding
author, phone: 1-416-736-2100 x20192; e-mail: allison@cse.yorku.ca).
978-1-4244-9270-1/15/$31.00 ©2015 IEEE
evaluated the accuracy of the Kinect sensor for static balance
assessment [5] and used the sensor to measure gait-related
parameters [6]. Notably, [7] used a Kinect sensor for
long-term, continuous, in-home fall detection.
Despite this work, it is clear that there is still a gap
between the measurement of gait-related parameters and an
automated gait assessment system. In addition, the results
given by a computer-based assessment system has to be
meaningful and convincing. Thus, it would be beneficial to
design a system that implements a standard clinical gait
assessment. The assessment results of such a system will be
more easily understood and accepted by clinics and patients.
Common clinical gait assessments include the Timed-up
and Go [8] and Tinetti assessment [9]. In [10], a 3-axis
accelerometer, wearable on ears (e-AR) was employed.
Algorithms were developed for assessing standing, walking
and sitting in Timed-up and Go assessment and for assessing
standing with eyes closed and 360º turning in the Tinetti
assessment. A limitation with this approach is that it is
difficult to calculate important gait parameters such as foot
clearance and step length by sensing only head accelerations.
These parameters are important for diagnosis and they may
reflect the progress of recovery of patients [11].
We focused on the Tinetti assessment because it covers a
wide range of dynamic balance analyses in different
scenarios. In general, the proposed system distinguishes itself
from previous work in its ability to measure the dynamics of a
patient's gait. The goal of the study is to set up reliable
standards for the computer-based gait assessment.
II. METHODOLOGY
Data Acquisition
Pre-processing
Segment Steps and Find Pauses of Feet Below
Calculate Trunk a Speed Threshold and
Parameters Calculate Trunk Parameters
Feature Extraction Feature Extraction
Classification Classification
Gait Analysis 360º Turning Analysis
Figure 1. Overview of the gait assessment
6679
 The system developed measures all aspects of the
standard Tinetti test in an automated, computer-guided
procedure, except the sub-assessment ‘Nudged’, based on
[12]. The present paper focuses on developing algorithms that
deal with the most challenging components of the Tinetti
assessment, namely the gait analysis and the 360º turning
analysis. Fig. 1 presents an overview of the system that
consists of data acquisition, pre-processing, gait analysis and
360º turning analysis.
A. Data Acquisition
The Kinect sensor samples at a frequency of approximately
30 Hz and video frames are captured both in color and depth.
Using captured frames, the middleware of Kinect SDK
segments and tracks human skeletons and gives the output of
a human skeleton represented by 20 nodes or control points in
the Kinect’s own reference frame known as the skeleton
space [13]. Each node represents a specific joint with 3D
position information in units of meters. The skeleton space
uses a right-handed coordinate system: the Y axis lies in
vertical direction of the image plane, the Z axis extends in
depth perpendicularly from the sensor and the X axis is
horizontal in the image plane and orthogonal to the Y and Z
axes [13].
B. Pre-processing
The position and the speed of each joint in the time
sequence are considered as one-dimensional signals. Two 2nd
order low-pass Butterworth smoothing filters were used to
reduce the noise in the signals. Empirically-determined
cut-off frequencies of 4Hz and 1Hz were used for the position
and speed signals of each joint, respectively.
C. Segmentation of Steps in Gait Analysis
To extract the features of walking steps, it is necessary to
accurately segment the steps, i.e. determine the start and the
end of a step. We used the Z component (in depth) of foot
speed because it showed good regularity in relation to the
phases of the steps. The algorithm robustly segmented the
steps while ignoring the small peaks generated by the
interference from parts of the body overlapping or the
distance between the subject and the camera being too long.
Fig. 2 illustrates an example time series of the Z speeds of
2.5
Left Foot
Right Foot
2 Feature Points
1.5
Speed (m/s)
1
0.5
0 0
0 50 100 150 200
Frames
Figure 2. Curves of Z speeds of two Feet in gait analysis
both feet during stepping. The most important features of the
curves are the start-, the mid- and the end-points. These are
identified as feature points and used for analyzing the gait.
The algorithm is insensitive to the tilt angle of the Kinect
sensor since we use the Z component (in depth) of foot speed
for step segmentation.
D. Finding Overlaps of Feet in 360º Turning Analysis
The 360º turning analysis uses the same pre-processing
step as the gait analysis. Since a subject is turning 360º on
spot, it is difficult to segment the steps using the method for
the gait analysis. To measure the continuity of a turn, the
algorithm identifies the skeleton frames in which the speeds
of both feet are below a certain speed threshold. Specifically,
the speed is defined as the Euclidean norm of X and Z
components of the speed of a foot. A group of consecutive
skeleton frames below a certain speed threshold indicates that
a subject may have paused during a 360º turn. As shown in
Fig. 3, the algorithm identified 6 pauses during the 360º turn
that are rendered in the shaded areas (based on a toe-off speed
threshold of 0.2 m/s). The time interval of each pause is
determined by the difference in timestamp of the first
skeleton frame and the last skeleton frame in a group.
E. Definition of Trunk Features
The stability of the trunk of the body was monitored by two
factors: the use of arms for balancing and the lean angle of the
trunk in the coronal plane. Additionally, for gait analysis, it is
necessary to calculate the deviation of the base of the spine
relative to the traveled path.
We assume that at the start of an assessment the subject is
not using the arms for balancing and the wrists are placed at
the sides of body (as directed by the therapist). In other words,
the wrists are at their resting positions. The distance between
wrists is defined as the Euclidean norm of X and Y
components of positions of two wrists. The Z component is
ignored since the arms typically swing during walking.
During a walk or a 360º turn, when subjects use their arms
for balancing or lean the trunk of their body, the distance
between the wrists will increase. By calculating the difference
of the distance between the wrists at the resting positions and
the distance of wrists during a walk or a 360º turn, the use of
0.45
Left Foot
0.4 Right Foot
0.35
0.3
Speed (m/s)
0.25
0.2
0.15
0.1
0.05
250 0 50 100 150 200 250
Frames
Figure 3. Curves of X-Z speeds of two feet in 360º turning analysis
6680
 1 Left Foot
Left Wrist Right Foot
0.8 Right Wrist

Speed (m/s)
0.6

1
Distance (m)

0.4
0
0.2 0
t2 sr
0
0.1 df
0.2 dl lr 200

Distance (m) t1 150

-0.2
-0.4
0 50 100
Frames
Figure 4. Uses of arms for balancing in 360º turning analysis
arms for balancing can be detected. To illustrate the process,
we draw the changes of X distances of two wrists with
respect to the origin of the Kinect in Fig. 4. The shaded area in
the figure indicates an interval in which a subject used the
right arm for balancing.
The leaning angle of the trunk is defined as the angle
between the vector of the trunk (between the center of
shoulders and the spine base) and the gravitational vector in
the coronal plane. The angle can be obtained by calculating
the dot product of these two vectors.
To measure the deviation from the path during a walk, a
path vector p⃑ is calculated using the position of spine base in
the first frame and last frame in a walk. The instantaneous
deviation from the path is defined as the perpendicular
distance between the position of the base of the spine and the
straight line path along vector p
⃑ as shown in Fig. 5.
F. Feature Extraction of Gait Analysis
For gait analysis, we are interested in the three feature
points of each step: the start, the mid and the end. Each
feature point contains the timestamp, the position and the
speed of the moving foot.
0.16
0.15
0.14
0.13
X Distance (m)
0.3 dr 100
50 Frames
0.4 0
150 200 250
Figure 6. The definition of gait features correlated with the movements of feet
(𝑑𝑓 denotes the distance between the heels and 𝑑𝑙 and 𝑑𝑟 denote the step lengths of
the left foot and the right foot, respectively)
We define the gait features involved in the gait analysis as
follows (with units in parentheses):
 Initiation of gait, 𝑡1 (𝑚𝑠);
 Step through length, 𝑙𝑙 (𝑚) and 𝑙𝑟 (𝑚);
 Step height speed, 𝑠𝑙 (𝑚 ⋅ 𝑠 −1 ) and 𝑠𝑟 (𝑚 ⋅ 𝑠 −1 );
 Step symmetry, 𝑑1 (𝑚);
 Step interval, 𝑡2 (𝑚𝑠);
 Path, 𝑑2 (𝑚);
 Trunk, 𝑑3 (𝑚) 𝑎𝑛𝑑 𝜃1 (°);
 Walking stance, 𝑑4 (𝑚);
Some of the features associated with the movements of feet
are illustrated in Fig. 6.
G. Feature Extraction of 360º Turning Analysis
We define the following features involved in the 360º
turning analysis:
 Continuity of steps 𝑡3 (𝑚𝑠);
 Steadiness 𝑑5 (𝑚);
Spine Base
H. Classification
Path Vector p The classification of normal and abnormal patterns of
each gait feature of a subject is performed by thresholding the
features extracted from the recorded skeleton. To determine
the thresholds of the features, data were captured from

0.12
athletes with potential risk of concussion and a kinesiologist
was asked to score the athletes by watching the prerecorded
0.11 videos using the software developed for the study. The scores
given by the kinesiologist were used as the ground truth for
0.1
determining the thresholds.
0.09

III. RESULTS
0.08
We designed the algorithms using Matlab 2014a for data
1.5 2 2.5 3 3.5 4 analysis. Using Microsoft Visual Studio 2013, we also
Z Distance (m) developed a desktop application for performing experimental
real-time assessments.
Figure 5. The deviation of spine base with respect to the traveled path

6681
 TABLE I
EXTRACTED FEATURES FOR GAIT ASSESSMENT AND 360º TURNING ASSESSMENT

Subject 𝑡1 𝑙𝑙 𝑙𝑟 𝑠𝑙 𝑠𝑟 𝑑1 𝑡2 𝑑2 𝑑3 𝜃1 𝑑4 𝑑5 𝑡3
1 1032 0.414 0.523 0.145 0.120 0.038 -164 0.001 0.087 0.817 0.007 0.128 32
2 1167 0.398 0.619 0.092 0.071 0.122 -132 0.019 0.112 2.303 0.020 0.090 164
3 1232 0.620 0.661 0.169 0.107 0.003 -332 0.030* 0.023 3.200 0.041 0.098 0
4 1104 0.432 0.474 0.080 0.060 0.014 -268 0.015 0.151* 2.808 0.025 0.156* 0
5 1232 0.379* 0.566 0.152 0.081 0.073 -200 0.008 0.098 0.998 0.038 0.086 36
6 1268 0.388 0.406* 0.196 0.112 0.123* -104 0.018 0.015 0.968 0.027 0.095 200*
7 392 0.474 0.475 0.036* 0.035* 0.048 -300 0.010 0.108 0.836 0.004 0.356 0
8 1204 0.465 0.464 0.222 0.052 0.033 -32* 0.023 0.039 2.754 0.013 0.098 0
9 1300* 0.407 0.505 0.105 0.121 0.079 -236 0.023 0.089 5.753* 0.059* 0.107 68
10 1263 0.467 0.456 0.047 0.048 0.057 -200 0.016 0.085 2.350 0.014 0.101 32
* The selected values for thresholds
TABLE II
THRESHOLDS FOR FEATURES OF GAIT ASSESSMENT AND 360º TURNING ASSESSMENT

𝑇𝑡1 𝑇𝑙𝑙 𝑇𝑙𝑟 𝑇𝑠𝑙 𝑇𝑠𝑟 𝑇𝑑1 𝑇𝑡2 𝑇𝑑2 𝑇𝑑3 𝑇𝜃1 𝑇𝑑4 𝑇𝑑5 𝑇𝑡3
1300 0.379 0.406 0.036 0.035 0.123 -32 0.030 0.151 5.753 0.059 0.156 200

The data were captured from 10 athletes by researchers in
the department of Kinesiology at York University. Three
athletes had a history of concussions, one had a suspected
concussion and the rest were healthy controls. Informed
consent was obtained from the participants in accordance
with a protocol approved by the Human Participants Review
Subcommittee at York University.
The base of the Kinect sensor was placed 0.84 m above
the ground. For gait analysis, the athletes were asked to
stand 3.8 m away from the camera, perform a walk towards
the camera and stop at 1.8 m away from the camera. For
360º turning analysis, the athletes were asked to stand at a
position between 1.8 m and 3.8 m away from the camera and
perform a 360º turn. To calibrate the system, we extracted the
features from the collected data using the developed Matlab
code, as shown in Table I. From the table, some interesting
patterns can be observed. For example, it took most subjects
more than 1000 ms to initiate a walk as shown in the second
column 𝑡1 .
Ideally, the sample data should cover all normal and
abnormal patterns of gait analysis and 360º turning analysis
so that it is possible to determine optimal thresholds for each
feature. However, the results of assessments conducted by the
kinesiologist showed that all participants received full scores
on all sub-assessments of gait analysis. All subjects but one
achieved full scores in the 360º turning analysis. This subject
used hands for balancing but the trunk was stable during the
360º turn.
The lack of abnormal results posed a problem for
determining thresholds for the assessments, since the data
samples were unable to cover all cases and the number of
samples was very limited.
Our approach to set the thresholds is to consider our sample
representative of normal variation. Then, for each feature, we
select the limit that will enable all normal participants to pass
the automated assessment (since they were passed by the
kinesiologist); the limit is normally the value that represents
the worst case in a sub-assessment as shown in Table II. For
example, in steadiness assessment in 360º turning analysis,
the value 0.1564 meter was selected instead of 0.3564 meter
because, in the latter case, the subject used arms for balancing,
which resulted in a longer distance. These thresholds are set
in our desktop application for experimental real-time
assessments. These parameter settings will form the initial
baseline for scoring abnormal gaits to be obtained in planned
clinical studies and subsequently refined to optimize
discrimination of normal and abnormal gait.
IV. DISCUSSIONS
The primary task for future study is to build a database that
contains as many samples as possible from relevant clinical
populations. When the number of samples is large enough
and adequately covers normal and abnormal patterns of each
gait feature, the accuracy of the segmentation will be
improved by determining a new threshold or using more
advanced classification algorithms. Further, thresholding
methods for classification might be improved upon by using
algorithms such as k-Nearest Neighbor, which can consider
the interaction of multiple features in determining gait.
A limitation of the gait analysis is the limited detection
range of the Kinect sensor. We have updated our desktop
application to support Kinect for Windows V2, which will be
used for future studies. The new Kinect sensor provides an
extended detection range for walking and better precision and
accuracy in tracking subjects.
In addition, we plan to compare our system using the
Kinect sensor with another motion tracking system using the
same set of features to determine the reliability of these
features.
V. CONCLUSION
We have presented a computer vision system based on the
methods revealed in [12]. The lack of not normal results and
very limited normal results did not permit us to rigorously
calibrate the system but did demonstrate its potential for
assessment of a subject’s mobility. We are still in the process
of capturing data and advanced classification algorithms may
be used when more data are available. In the future, we expect

6682
the system would be suitable for clinical or locker room
assessment of gait impairment due to concussion. Gait
assessment is important in a variety of other neurological
contexts and we believe the system may have utility assessing
or tracking the impact and recovery from other illness,
injuries and disorders.
REFERENCES
[1] B. Toro, C. Nester, and P. Farren, “A review of observational gait
assessment in clinical practice,” Physiotherapy Theory Practice, vol. 19,
pp. 137–149, 2003.
[2] A. D. Kloos, V. D. Bello-Haas, R. Thome, J. Cassidy, L. Lewis, T.
Cusma, and H. Mitsumoto, “Interrater and Intrarater Reliability of the
Tinetti Balance Test for Individuals with Amyotrophic Lateral
Sclerosis,” Journal of Neurologic Physical Therapy, vol. 28, no. 1, pp.
12–19, Mar. 2004.
[3] M.-R. Lin, H.-F. Hwang, M.-H. Hu, H.-D. Wu, Y.-W. Wang, and F.-C.
Huang, “Psychometric Comparisons of the Timed Up and Go, One-Leg
Stand, Functional Reach, and Tinetti Balance Measures in
Community-Dwelling Older People,” Journal of the American
Geriatrics Society, vol. 52, no. 8, pp. 1343–1348, Aug. 2004.
[4] L. M. Cipriany-Dacko, D. Innerst, J. Johannsen, and V. Rude,
“Interrater reliability of the tinetti balance scores in novice and
experienced physical therapy clinicians,” Archives of Physical
Medicine and Rehabilitation, vol. 78, no. 10, pp. 1160–1164, Oct.
1997.
6683
[5] Yang Yang, Fang Pu, Yan Li, Shuyu Li, Yubo Fan, and Deyu Li,
“Reliability and Validity of Kinect RGB-D Sensor for Assessing
Standing Balance,” IEEE Sensors Journal, vol. 14, no. 5, pp. 1633-1638,
May 2014.
[6] R. A. Clark, K. J. Bower, B. F. Mentiplay, K. Paterson, and Y.-H. Pua,
“Concurrent validity of the Microsoft Kinect for assessment of
spatiotemporal gait variables,” Journal of Biomechanics, vol. 46, no. 15,
pp. 2722–2725, Oct. 2013.
[7] Stone, E.E., and Skubic, M., “Unobtrusive, Continuous, In-Home Gait
Measurement Using the Microsoft Kinect,” IEEE Transactions on
Biomedical Engineering, vol. 60, no. 10, pp. 2925-2932, Oct. 2013.
[8] Podsiadlo D., and Richardson S., “The timed “Up & Go”: a test of basic
functional mobility for frail elderly persons,” J Am Geriatr Soc., vol. 39,
no.2, pp.142-148, Feb. 1991.
[9] Tinetti, M.E., Williams, T.F. and Mayewski, R., “Fall risk index for
elderly patients based on number of chronic disabilities,” American
Journal of Medicine, vol. 80, no. 3, pp. 429–434, 1986.
[10] King, R.C., Atallah, L., Wong, C., Miskelly, F., and Yang, G.-Z.,
“Elderly Risk Assessment of Falls with BSN,” 2010 International
Conference on Body Sensor Networks (BSN), pp. 30-35, June 2010.
[11] von Schroeder HP, Coutts RD, Lyden PD, Billings E Jr, and Nickel VL,
“Gait parameters following stroke: a practical assessment,” J Rehabil
Res Dev., vol. 32, no. 1, pp. 25-31, Feb. 1995.
[12] Bunn, F.E., and Adair, R.D., “Assessment and cure of brain concussion
and medical conditions by determining mobility,” USPTO patent
pending App. No. 13/550,930 filed July 17, 2012.
[13] https://msdn.microsoft.com/en-us/library/hh973078.aspx, last accessed
June, 2015.