Professional Documents
Culture Documents
Using Lucas-Kanade Algorithms To Measure Human Movement
Using Lucas-Kanade Algorithms To Measure Human Movement
Human Movement
1 Introduction
This paper described the life cycle of the whole project, including the preliminary study
of motion estimation, the basic implementation of Lucas-Kanade algorithm, the orig-
inal testing results of Lucas-Kanade algorithm, the testing results of Lucas-Kanade
algorithm after improving, the output of improved algorithm, and the evaluation, as
well as analysis, of final data.
rI v ¼ It : ð1Þ
Where rI ¼ Ix ; Iy is the spatial intensity gradient, v ¼ ðu; vÞ is the image
velocity, and It is the change of intensity with respect to time (Beauchemin and Barron
1995).
Optical flow refers to the movement speed of each pixel in the image. In order to
calculate the optical flow, the intensity of each object is assumed to be constant, thus,
given that point Aðx; yÞ moves to Bðx þ dx; y þ dyÞ on dt, the intensity of this point is
uniform. Therefore,
Where Iðx; y; tÞ is the intensity of point Aðx; yÞ on time t, Iðx þ dx; y þ dy; t þ dtÞ it
the intensity of this point on time t þ dt, while dx and dy stand for the displacement of
this point on horizontal and vertical aspects respectively.
Moreover, according to Taylor expansion,
@I @I @I
I ðx þ dx; y þ dy; t þ dtÞ ¼ I ðx; y; tÞ þ dx þ dy þ dt þ H:O:T: ð3Þ
@x @y @t
If the displacement of the point and the time are sufficiently small, H:O:T (Higher
Order Derivative) can be ignored. Therefore,
@I @I @I
dx þ dy þ dt ¼ 0: ð4Þ
@x @y @t
@I dx @I dy @I
þ þ ¼ 0: ð5Þ
@x dt @y dt @t
IX u þ Iy v þ It ¼ 0: ð6Þ
IX u þ Iy v ¼ It : ð7Þ
Under this assumption, an optical flow, which maps the approximated motion
between two frames of a video, can be drawn onto a 2D plane.
However, this is an ill-posed equation, since there are two unknowns in one
equation. And many researchers try to solve this problem, and finally, come up with
solutions to deal with this problem. More specifically, methods to solve optical flow
120 Y. Mi et al.
computation can be further divided into the local method, also known as Lucas-Kanade
technique or Bigun’s structure tensor method, and global method, including
Hom/Schunck approach and its extensions (Bruhn et al. 2005).
Av ¼ b: ð10Þ
Applied the least squares method to solve this overdetermined equation, thus:
AT Av ¼ AT b: ð11Þ
1
v ¼ AT A AT b: ð12Þ
Using Lucas-Kanade Algorithms to Measure Human Movement 121
Fig. 1. Optical flow field (c) displaying box rotation to the right from (a) to (b)
2 Related Work
2.1 Improvements About Motion Estimation
In order to solve the problems about the motion of the camera and the correspondence
of all pixels, Irani and Anandan (1999) proposed a direct method, which can collect all
the information of the pixels in the image, to measure the target’s movement. However,
a direct method is sensitive to natural noises, and will cost a long time to compute
(Zisserman 1999). To improve the processing speed, Nguyen and Tan (2004) proposed
a series of fast block-matching algorithms (BMAs), which is used to limit the number
of search locations or to simplify the measure of match between two blocks under
comparison.
However, fast block-matching algorithms (BMAs) method cannot accurately figure
out the global motion in other camera motion, like scaling or rotating. To solve this
problem, Huang and Hsieh (2004) come up with a feature-based method. Additionally,
Niu et al. (2014) also proposed a fast and robust local method, based on the Lucas-
Kanade paradigm, to process large image formats by removing false feature point.
3 Solution
3.1 Tools
Matlab was used to implement the algorithm and draw the optical flow field according
to the outputs, because, as a widely used method in image processing and video
manipulation area, Matlab is a clear language and can be documented easily (Gonzalez
et al. 2006). Additionally, the agile approach was adopted in this project, as, compared
with other methods, like waterfall module, it can be changed or modified easily (Cao
and Ramesh 2008).
Moreover, the source of the videos used in this project were taken from Medtronic
study, and the videos were created under a lab setting using the Kinect. Additionally,
Using Lucas-Kanade Algorithms to Measure Human Movement 123
four different types of videos (video_1 – before painkillers with crutches, video_2 –
before painkillers without crutches, video_3 – after painkillers with crutches, video_4 –
after painkillers without crutches), which show a man with chronic pain walking a few
yards, turning around and walking back, have been analyzed.
4 Results
In terms of patients’ movement, Fig. 3(a) and (b) cannot show the moving trail
clearly, while Fig. 3(c) and (d) give a clear moving trail. Comparing with Figs. 3(c)
and (d) has fewer arrows, while it can also display the movement trail effectively.
In addition, processing speed is another indicator to evaluate the whole system.
Table 1 shows the average magnitude of different algorithms.
From this table, it can be seen that, after applying feature points, threshold, and
resizing method, the time cost for processing each frame decreased significantly.
In this interface, users can simply choose one implementation and then set the
parameter according to their needs. After clicking ‘OK’ button, the outcome, including
the motion trial and related data would be displayed on the screen directly.
Using Lucas-Kanade Algorithms to Measure Human Movement 127
5 Evaluation
Just as previous analysis in Sect. 4, all the project aims, including functional and non-
functional aims have been met, thus, generally, the solution is successful. However, to
further evaluate the system and for future use, the strengths, limitations, and approach
of this project would be examined in this section.
6 Conclusions
6.1 Project Overview
During this project, a system using basic Lucas-Kanade algorithm and its three
improvements was developed. And, due to the accuracy and speed, Lucas-Kanade
implementation with feature points, threshold and resizing was chosen to analyze the
four videos. In addition, a simple but clear interface has been developed for general
users, and the output would be transformed into easily understandable data to users.
References
Alex, D.S., Wahi, A.: BSFD Background subtraction frame difference algorithm for moving
object detection and extraction. J. Theor. Appl. Inf. Technol. 60(3), 623–628 (2014)
Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework. Int. J. Comput. Vis.
56(3), 221–255 (2004)
Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets horn/Schunck: combining local and
global optic flow methods. Int. J. Comput. Vis. 61(3), 1–21 (2005)
130 Y. Mi et al.
Cao, L., Ramesh, B.: Agile requirements engineering practices: an empirical study. IEEE Softw.
25(1), 60–67 (2008)
Ding, L., Goshtasby, A.: On the canny edge detector. Pattern Recogn. 34(3), 721–725 (2001)
Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modelling and computation: a survey.
Comput. Vis. Image Underst. 134, 1–21 (2015)
Gibson, J.J.: The visual perception of objective motion and subjective movement. Psychol. Rev.
101(2), 318–323 (1994)
Gonzalez, R.C., Woods, R.E., Eddins, S.L.: Digital image processing using MATLAB. Dorling
Kindersley Pvt, India (2006)
Heath, M., Sarkar, S., Sanocki, T., Bowyer, K.: Comparison of edge detectors. Comput. Vis.
Image Underst. 69(1), 38–54 (1998)
Huang, J.-C., Hsieh, W.-S.: Automatic feature-based global motion estimation in video
sequences. IEEE Trans. Consum. Electron. 50(3), 911–915 (2004)
Huang, T.-C., Chang, C.-K., Liao, C.-H., Ho, Y.J.: Quantification of blood flow in internal
cerebral artery by optical flow method on digital subtraction angiography in comparison with
time-of flight magnetic resonance angiography. PLoS ONE 8(1), e54678 (2013)
Irani, M., Anandan, P.: About direct methods. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.)
IWVA 1999. LNCS, vol. 1883, pp. 267–277. Springer, Heidelberg (2000). https://doi.org/10.
1007/3-540-44480-7_18
Iwata, K., Kikuta, H.: Measurement of dynamic flow field by optical computed tomography with
shearing interferometers. Opt. Rev. 7(5), 415–419 (2000)
Ko, B., Kim, S., Nam, J.: Image resizing using saliency strength map and seam carving for white
blood cell analysis. Biomed. Eng. Online 9(1), 54 (2010)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo
vision. In: IJCAI, pp. 674–679 (1981)
Lu, J., Liou, M.L.: A simple and efficient search algorithm for block-matching motion estimation.
IEEE Trans. Circ. Syst. Video Technol. 7(2), 429–433 (1997)
Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion
capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–126 (2006)
Nguyen, V.A., Tan, Y.: Fast block-based motion estimation using integral frames. IEEE Signal
Process. Lett. 11(9), 744–747 (2004)
Niu, Y., Xu, Z., Che, X.: Dynamically removing false features in pyramidal Lucas-Kanade
registration. IEEE Trans. Image Process. 23(8), 3535–3544 (2014)
Torr, P.H.S., Zisserman, A.: Feature based methods for structure and motion estimation. In:
Triggs, B., Zisserman, A., Szeliski, R. (eds.) IWVA 1999. LNCS, vol. 1883, pp. 278–294.
Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44480-7_19
Singla, N.: Motion detection based on frame difference method. Int. J. Inf. Comput. Technol. 4
(0974–2239), 1559–1565 (2014)
Wang, Z., Ben Salah, M., Zhang, H.: Object joint detection and tracking using adaptive multiple
motion models. Vis. Comput. 30(2), 173–187 (2013)