D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0 PDF

D4.7.
3 Experiment Results and Recommendation

2013-10-29 Jos Manuel Jimnez (STT Ingeniera y Sistemas)
This document presents the results of the experiments carried out at CAR Venue (Sant Cugat) in the framework of the 3D Acrobatics experiment. Three different runs of the experiment were carried out between May 2103 and September 2013. In two of these runs the experiment collaborated with other experiments within EXPERIMEDIA project, namely CONFetti experiment and CAR experiment. The results collected in those runs are discussed and evaluated. Key conclusions extracted from the results related to the use of FMI in this context are highlighted. This document has been prepared considering particularly information contained in deliverables D2.1.1 (First EXPERIMEDIA Methodology), D4.7.1 (3D Acrobatics: experiment description and requirements), D2.2.1 (EXPERIMEDIA Baseline Components), D4.2.1 (CAR Experiment Design and Plan) and D4.7.2. (3D Acrobatics: Experiment Progress Report).
www.experimedia.eu
EXPERIMEDIA
Project acronym EXPERIMEDIA
Dissemination Level: PU
Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.7 EX7 3D Acrobatics Deliverable lead organisation STT Ingeniera y Sistemas Authors Jos Manuel Jimnez Bascones, (STT Ingeniera y Sistemas) Reviewers Sandra Murg (JRS) Version 1.0 Status Final Dissemination level PU: Public Due Date PM24 (2013-09-30) Delivery Date 2013-10-29
Copyright STT Ingeniera y Sistemas and other members of the EXPERIMEDIA consortium 2013
EXPERIMEDIA
Table of Contents
1. 2. 3. Executive Summary............................................................................................................................ 3 Introduction ........................................................................................................................................ 4 Description of the Experiment ........................................................................................................ 5 3.1. 3.2. 3.3. General Description of the 3D Acrobatics Experiment ..................................................... 5 Overview of the Experiment Design ..................................................................................... 6 Components Developed within the 3D Acrobatics Experiment ...................................... 8
3.2.1. System architecture ............................................................................................................... 7 3.3.1. Motion capture ...................................................................................................................... 8 3.3.2. Generation of video contents ........................................................................................... 13 3.3.3. Synchronization with metadata ......................................................................................... 13 3.3.4. Visualization of motion capture data for training purposes ......................................... 14 3.3.5. Implementation of VRPN server ..................................................................................... 14 3.4. 3.5. 4. 4.1. 4.2. 4.3. 5. 5.1. 5.2. 5.3. 6. Equipment................................................................................................................................ 15 Ethics and privacy ................................................................................................................... 16 First experiment run ............................................................................................................... 18 Second experiment run .......................................................................................................... 19 Third experiment run ............................................................................................................. 21 Motion capture experience .................................................................................................... 24 Lessons learnt from FMI ....................................................................................................... 25 Dissemination .......................................................................................................................... 26
Experiment execution ...................................................................................................................... 18
Results ................................................................................................................................................ 24
Conclusions ....................................................................................................................................... 27
EXPERIMEDIA
1. Executive Summary
This document describes the results obtained in the execution of the 3D Acrobatics experiment during the development period. A general overview of the whole experiment is reported describing in detail the technical goals achieved. A detailed explanation of the activities performed on each task within the experiment is also presented. The results obtained in the execution of the experiment are outlined and the main conclusions collected in the different runs of the experiment are highlighted. The exploitation plan is included in the last section of the document. The report is organized in several sections starting by this executive summary in Section 1. Section 2 includes a brief introduction to the 3D Acrobatics experiment and its relationship with Future Media Internet products and services. Section 3 presents an overall description of the experiment. In this section a detailed explanation of the different software modules implemented in this experiment is included. Ethics and privacy aspects of the experiment are also discussed in this section. Section 4 reports the execution of practical runs of the experiment carried out at CAR Venue. Some of these runs we performed in collaboration with other experiments within the EXPERIMEDIA project while other runs were executed exclusively between CAR and STT. Section 5 discusses the results obtained in the different runs of the experiment paying special attention to the outstanding elements considered in this development such as the use of motion capture technologies in real training conditions with high performance athletes, the integration of the motion capture data with FMI products and services, the impact of these new services in the target community, etc. This section ends with a summary of the lessons learnt in the practical implementation of this experiment. Section 6 summarizes the conclusions obtained in this experiment. Section 0 presents a summary the next actions that STT will undertake in order to further develop this product; in addition, the exploitation plan for the resulting new product is outlined. This document completes the information included in documents D4.7.11 and D.4.7.22, which are the previous deliverables corresponding to the 3D Acrobatics experiment.
D4.7.1 3D Acrobatics: experiment description and requirements https://svn.it-innovation.soton.ac.uk/svn/experimedia-docs/Official deliverables/D4.7.1 3D Acrobatics Experiment Description and Requirements v1.0.pdf 2 D4.7.2 3D Acrobatics: experiment progress report https://svn.it-innovation.soton.ac.uk/svn/experimedia-docs/Official deliverables/D4.7.2 3D Acrobatics Experiment Progress Report v1.0.pdf
1
EXPERIMEDIA
2. Introduction
EXPERIMEDIA is a collaborative project aiming to accelerate research, development and exploitation of innovative Future Media Internet (FMI) products and services through test beds that support experimentation in the real world which explore new forms of social interaction and experience in online and real world communities. Within this framework 3D Acrobatics experiment focuses on high quality content production for assessment and improvement in gymnastic exercises by the use of motion capture technologies. The goal of this experiment is to record training sessions of gymnastics at CAR Venue and automatically generate assessment data for helping the athletes improve their performance. These 3D motion capture data is also used to compute metadata which is synchronized and saved with the athletes motion in order to provide a valuable 3D graphics and augmented reality experience. The experiment makes use of the connectivity and storage facilities available at CAR Venue. 3D Acrobatics integrates 3D graphics as well as augmented reality and synchronization with external video. The experiment carried out research on synchronization of 3D motion capture, video and metadata. The experiment makes emphasis on the quick delivery of data to the athletes and trainers. To this end it makes use of mobile devices (such as tablets, laptops, etc.) in order to collect data from the inertial sensors (as well as from other motion capture devices and technologies) and provides feedback to the users in real-time on almost any platform (computers, tablets, laptops, smartphones, etc.). The information gathered by the motion capture devices is transferred to a local computer, laptop or tablet. This device stores locally the motion capture data and connects through a Wi-Fi connection to the server. A data manager (software module) manages this information in the cloud making it available to the community. In this way the athlete has the possibility of sharing her/his data with trainers, colleagues and mates who might be geographically distributed, thus enlarging her/his experience in training and gymnastics. Recording sessions have been run at CAR Venue in order to evaluate the developments done within the 3D Acrobatics experiment. Figure 1 shows examples of the training sessions recorded at CAR during the different runs of the experiment.
Figure 1. Experiment runs at CAR.
EXPERIMEDIA
3. Description of the Experiment

3.1. General Description of the 3D Acrobatics Experiment
The 3D Acrobatics experiment focuses on high quality content production for the gymnastic training sessions including 3D motion capture based on inertial sensors and 3D biomechanical analyses.
Figure 2. Inertial sensor STT-IBS.
The use of inertial sensors for motion capture in training sessions for gymnastics and other sports can be an important improvement for the assessment and training of athletes. Thanks to its reduced size (see Figure 2) inertial sensors can be easily attached to the athletes body without compromising her/his mobility in anyway.
Figure 3. Inertial sensors attached to the athlete's body.
Another important characteristic of these sensors is that each one includes its own Bluetooth antenna which allows it to connect directly to the device hosting the antenna. This feature facilitates dramatically the task of fixing the sensors on the athletes body since no wires are required at any moment (see Figure 3).
Copyright STT Ingeniera y Sistemas and other members of the EXPERIMEDIA consortium 2013 5
EXPERIMEDIA
3D Acrobatics carried out research on synchronization of motion capture data gathered from the inertial sensors, video obtained from the cameras available at the CAR Venue and metadata. The experiment aimed at focussing on the use of all those elements in the training process and on the improvement of the athlete's technique. In the event that the training session was recorded using stereoscopic (3D) HD cameras these contents is also synchronized with the motion capture data. To this end a VRPN3,4 (Virtual Reality Peripheral Network) server has been implemented allowing the communication between the software controlling the stereoscopic cameras and the motion capture data. This feature has been successfully tested in collaboration with the EXPERIMEDIA experiment CONFetti. As the experiment runs took place in the CAR Venue the interaction between athletes, trainers and other professionals involved in the preparation of the athletes was very effective during the practical execution of the experiment. Mobile devices such as tablets or laptops were used to collect the data from the inertial sensors. In addition laptop computers were used to collect motion capture data using optical systems. These alternatives are discussed in detail in Section 4. Motion capture data was displayed on those devices in real-time providing in this way instantaneous feedback to trainers; athletes benefited as well of this display since they got feedback immediately after the exercise was performed. Examples of the use of the motion capture devices are reported in Section 4. 3D motion data was collected by a computer connected to motion capture devices. This computer included a data manager which allowed uploading the motion capture information to a repository in the server. This data was retrieved and analysed in order to generate metadata that could be used for improving the technique of the athletes by the use of advanced 3D graphics. Finally the 3D motion capture data and metadata were stored in the cloud. The data manager module allows the athlete to access her/his data and share it with trainers, colleagues and friends who might be geographically distributed. In this way the athlete is able to share the experience with a large and distributed community. The athlete can use this way to get advice from other trainers as well as remarks from other athletes.
3.2.
Overview of the Experiment Design
The implementation of the 3D Acrobatic Sports experiment required the integration of a motion capture system (developed by STT) with the SOA5 (Service-Oriented Architecture) implemented by ATOS to manage the data recorded at CAR Venue. In addition, 3D video and motion capture data have been synchronized using a VRPN server; this task has been carried out in collaboration with CONFetti experiment.
Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber, Jeffrey Juliano, Aron T. Helser, "VRPN: A Device-Independent, Network-Transparent VR Peripheral System", VRST 01, November 15-17, 2001, Banff, Alberta, Canada. Copyright 2001 ACM 1-58113-427-4/01/0011 4 VRPN: Virtual Reality Peripheral Network, http://www.cs.unc.edu/Research/vrpn/ 5 David Sprott and Lawrence Wilkes, "Understanding Service-Oriented Architecture", CBDI Forum, January 2004.
3
EXPERIMEDIA
As indicated in the previous paragraph, 3D Acrobatics experiment also interacts with the software developments made by ATOS in the CAR experiment. The same SOA is used to manage the data which is collected in the motion capture sessions as well as the metadata generated when the motion capture sessions are reviewed. Therefore an effective collaboration with the CAR experiment has also been carried out.
3.2.1.
System architecture
Figure 4 depicts graphically the system architecture implemented in the 3D Acrobatics experiment.
Figure 4. System architecture.
The functional building blocks of this architecture are the following: Inertial (or other technology) motion capture system Generation of video contents Synchronization of motion capture data with video and metadata Visualization of motion capture data for training purposes VRPN server
The actual implementations and/or modifications of these building blocks are described in detail in Section 3.3. From the AVCC component the implemented by ATOS the SOA functions have been used in the 3D Acrobatics experiment. For the generation of metadata and synchronization a new software application has been developed by STT.
EXPERIMEDIA
3.3.
Components Developed within the 3D Acrobatics Experiment
The components or building blocks developed within the 3D Acrobatics experiment are summarized in the following list: 1) Adaptation of the inertial motion capture application. This task focused on the modification of the software modules that solve the problem of tracking human motion using wireless inertial sensors. Specific biomechanical models for gymnastics have been implemented and used in the experiment runs at CAR Venue. The main modifications aimed at the implementation of protocols for analysis of pommel horse exercises as well as jump analysis. Due to some limitations identified in the use of inertial sensors during the experiment runs it was decided to undertake similar developments in order to use optical motion capture technologies in gymnastic exercises as well as other sports disciplines. 2) Generation of video contents. The objective of this task was the generation of a video file from the 3D engine integrated in the capture application. Originally it was planned to integrate this modification in the motion capture software for inertial sensors. Due to the fact that optical motion capture systems were used as well the same modifications related to video generation were included in the motion capture software with optical technology. The files generated in this step are uploaded to a repository using the SOA implemented by ATOS for the experiment being performed at CAR. The SOA have been used in the implementation of a data manager module developed within this task. 3) Synchronization of motion capture data with video and metadata. In this task a software module that allows generating metadata with video has been implemented. This software allows the synchronization of the metadata with the video generated by the motion capture software. Although it was originally planned to use the software modules developed by ATOS for CAR Venue, this software module was finally developed by STT. 4) Visualization of motion capture data for training purposes. The goal of this task is the implementation of a 3D visualization module which runs on the devices hosting the Bluetooth connection with the inertial sensors; the same module has been implemented for the optical motion capture solution. This visualization module allows trainers and athletes to visualize athletes motion. It provides instantaneous feedback to the athlete once she/he concludes the execution of her/his exercise. 5) Implementation of VRPN server. This block has been implemented in order to allow synchronization between the motion capture data with the stereoscopic video recorded in the CONFetti experiment. All the components described in the previous list were used in recording sessions carried out at CAR Venue during the execution of the project. The results of these runs are described in Section 4.
3.3.1.
Motion capture
The motion capture system was a key element in the development of the proposed experiment. This software application integrates three main functions: 1) Data collection. This is a function of the motion capture software application which is able to connect to a given set of devices in order to collect the motion data corresponding to the
EXPERIMEDIA
athlete being analysed. If inertial sensors are used the raw data is collected using the triaxial sensors integrated on the sensor's board. These sensors are a triaxial magnetometer, a triaxial accelerometer and a triaxial gyroscope. These triaxial sensors provide a set of nine degrees of freedom (raw data) which are converted into three-dimensional rotations by using a Kalman filter. If optical technology is used the raw data collected from the cameras is composed of a set of 2D coordinates which represent the projection of the optical markers on the camera sensor. By using the camera calibration data (extrinsic and intrinsic parameters) the set of 2D coordinates are transformed into a set of 3D points. These are the coordinates of the markers in the 3D space. As long as either the inertial sensors or the optical markers are located in predefined anatomical positions of the athlete the raw data can be used to compute biomechanical parameters by using the corresponding biomechanical models. 2) Biomechanical models. The second function of the motion capture application is the biomechanical module. This software component is in charge of analysing the data collected from either the inertial sensors or the optical markers and translating it into a coherent kinematic skeleton of the human body. This kinematic model is in turn used to compute the biomechanical parameters relevant for the type of analysis that the trainer or athlete wants to perform. 3) Motion analysis. The third and last function of the motion capture application is a software module which automatically saves the motion data file and generates a report for the captured motion. This module generates a 3D animation of the captured motion using a skeleton-like avatar. In addition this module can show the computed biomechanical data in the form of graphs and reports. The 3D Acrobatics experiment required the implementation of particular biomechanical models in the inertial motion capture solutions developed by STT. In particular protocols for pommel horse analysis and jump analysis have been implemented in this application. The details of the biomechanical models implemented in this software application are fully described in the deliverable D4.7.2. The detailed analysis of the results collected in the experiment runs carried out at CAR Venue in May 2013 and July 2013 revealed some limitations on the use of inertial motion capture technologies in pommel horse analysis. For this reason it was decided to include in the experiment the use of optical motion capture technologies in order to overcome the identified limitations. These limitations are described in detail in Section 4. This decision imposed the need to implement in the software application for optical motion capture modifications similar to those implemented in the inertial motion capture application in the first half of the experiment. Therefore the optical motion capture system was modified in order to implement protocols which allowed data collection, included biomechanical models and performed motion analysis of the gymnastic exercises (pommel horse and jumps) using optical markers. In addition, cycling analysis was also considered in order to allow a comparative analysis with the cycling experiment driven by CERTH at CAR Venue.
EXPERIMEDIA Data collection
For the motion capture approach based on the use of inertial sensors one the main developments made within this project has been the implementation of the Kalman filter in the processor built-in the printed board circuit (PCB). The advantages derived from this modification are the optimization on the use of the bandwidth for data transmission as well as the possibility to increase the sample rate of the sensor up to 250 Hz. A secondary action developed within this task is the development of a library which allows the connection of the inertial sensor with a Bluetooth antenna running on Linux or Android operating systems. This library is currently under testing. This task is encompassed in the future actions and developments being undertaken by STT after completion of the 3D Acrobatics experiment. Finally the calibration of the internal sensors in the inertial sensor has also been improved in order to take into account the exigent conditions imposed by the motions in the exercises executed in pommel horse and other gymnastic specialities. For the motion capture approach based on the use of optical markers the data collection module has been modified in order to include a marker set which allows capturing optical raw data for gymnastic motions such as pommel horse exercises or jump analysis. Analogously a marker set for cycling analysis has also been implemented. The aim of this marker set was to perform comparative analysis with the Kinect experiment driven by CERTH at CAR Venue. Biomechanical models The goal of this section was the implementation of a set of biomechanical models which provide the data required for the assessment of the training process in acrobatic sports. The focus of this task was the analysis of pommel horse exercises as defined in deliverable D4.7.1. The data collected by the biomechanical models describe in an objective and quantifiable manner the performance of the athlete carrying out some exercises. After analysis and review of these data the trainers are able to propose and recommend to the athlete the best practices that will lead to a significant improvement in the execution of the exercises. In addition the athletes are able to visualize these objective and quantifiable data by using 3D graphics which helps them dramatically to understand the explanations and recommendations made by the trainers. Within 3D Acrobatics experiment two solutions have been implemented for quantitative analysis of the athlete's motion. On one hand a solution based on the use of the inertial sensors was proposed. The biomechanical parameters used to evaluate the athlete's performance are computed from the measurements collected by the inertial sensors taking into account the topological information provided by the biomechanical model. The inertial sensors used on each biomechanical model must be located on the right anatomical positions of the athlete's body in order to allow computing of the required data. Inertial sensors are light and comfortable to wear by the athlete. They do not impose any limitation to the motions of the individual being analysed.
EXPERIMEDIA
Figure 5. Relative angles measured in anatomical joints using two inertial sensors.
In biomechanics, the magnitudes of interest are very often defined as relative angles measured in the anatomical joins. In order to compute those relative angles, the absolute rotations of two inertial sensors placed at the limbs joined by the anatomical articulation are used. From the absolute rotations of the two inertial sensors the relative orientation between them can be easily computed. This relative orientation allows computing relative angles at the joint such as internal/external rotation, flexion/extension and abduction/adduction (see Figure 5). In the scope of the 3D Acrobatics experiment the main gymnastic exercise selected for the experiment validation was pommel horse. The pommel horse is one of the six pieces of equipment in men's artistic gymnastic competitions. A contemporary pommel horse exercise is characterized by different types of circles with the legs apart or joined in a variety of support positions on all parts of the horse. The performance, as assessed by the judges who judge the exercise and determine the sum of the technical and positional execution errors, depends on the gymnast's ability to perform the circle with maximum amplitude. For angles two main variables are identified: the angle formed by the truck and arms and the angle formed by the trunk and legs (see Figure 6). In addition to pommel horse a protocol for jump analysis was implemented.
Figure 6. Two positions during pommel horse exercise.
The analysis of the specialized literature led to the definition of a set of protocols for the analysis of pommel horse exercises using inertial sensors. These protocols are described in detail in deliverable D4.7.2. On the other hand a second solution for quantitative motion analysis was proposed based on the use of optical motion capture technology. The need of use of optical motion capture required the implementation of protocols similar to those described in previous paragraphs using optical
EXPERIMEDIA
markers. Optical motion capture technologies differ from the technology based on inertial sensors in the type of data captured by the motion capture devices. While inertial sensors provide 3D rotations of the body segments they are attached to, optical motion capture technology gives the 3D coordinates of the optical markers. These are small light spheres covered by reflective tape. The basic principle behind optical motion capture systems is the fact that the markers reflect the infrared light emitted by the LEDs located around the camera lenses. This reflected light is seen by the motion capture cameras which transmit to the computer the 2D coordinates of the reflections in the reference frame of each image sensor. Using the calibration parameters these 2D coordinates are used to compute the 3D coordinates of the markers in the global (calibration) reference frame. In order to compute the position and orientation of a body segment in the space it is required that the segment has at least three optical markers attached to it. In the sought of keeping the total number of optical markers moderate for a given protocol the optical markers are usually located at the body joints. In this way a given optical marker is shared between two contiguous segments and its coordinates are used to compute the positions and orientations of the two segments linked together by the anatomical joint. Figure 7 depicts an example of an optical marker set for cycling analysis.
Figure 7. Optical marker set for cycling analysis.
Optical markers are very light and small. The use of optical markers in practical training sessions does not impose any restriction to the athlete who can perform her/his exercises without any problem. Wearing the markers on the body is not uncomfortable. During the development of the 3D Acrobatics experiment adequate protocols have been developed in order to allow using optical motion capture technology during practical training sessions. Motion analysis The module of motion analysis has been modified according to the characteristics of the biomechanical models implemented in the previous section. This module allows representing the motion of the body segments captured by the described models using a skeleton-like avatar. In
EXPERIMEDIA
addition the biomechanical parameters computed by the implemented models are shown in form of 2D plots in a separate window. These modifications have been implemented in both the inertial and the optical motion capture solutions.
3.3.2.
Generation of video contents
The motion capture applications (both inertial and optical) include a 3D render engine which is used to visualise the motion executed by the athlete using a skeleton-like avatar. A new function was implemented in this 3D graphics engine aimed at generating a video sequence for the animated avatar. This function allows the use of the 3D engine to select a point of view in the 3D window and to generate a video sequence of the selected time interval of the exercise being analysed. This video is saved in a file in the hard drive in H.264 format. The file containing the video generated can be uploaded to the repository using the data manager application developed for this purpose using the SOA implemented by ATOS. Therefore the repository can contain video files as well as the motion capture files. All these files can be retrieved for further analysis either by the athlete or the trainer.
3.3.3.
Synchronization with metadata
The main goal of this task was the implementation of a software module which allows the synchronization of the metadata with the motion capture data and video (generated or recorded). Originally, it was foreseen that this software was going to use the software tools developed by ATOS for the CAR experiment. These tools included both the SOA for handling files and a graphic user interface with functions aimed at creating annotations (metadata) in the video frames. The software module for synchronization implemented by STT makes use of the SOA but implements a set of functions for video annotation totally implemented by STT. Figure 8 depicts a view of the user interface of the application which allows defining metadata on video generated by the motion capture applications. In order to synchronize motion capture data and video (either recorded by GigE cameras of stereoscopic 3D HD cameras) a trigger function was implemented in both the optical and the inertial motion capture applications. This trigger function was implemented to support two different operation modes. On one hand the motion capture applications are able to send a trigger signal when the motion capture process is started. This signal can be captured by the software running the recording process with the cameras and at this moment the cameras can start recording video. On the other hand the motion capture applications have been modified so as to accept a trigger signal launched by another component. This signal is captured by the motion capture applications and the motion capture process is launched when the signal is received.
13
EXPERIMEDIA
Figure 8. Application for synchronization of video and metadata.
The implemented functions can be used by ATOS for the purpose of synchronizing video recorded by the GigE cameras available at CAR and motion capture data recorded using the inertial sensors. Analogously PSNC can make use of this function within the CONFetti experiment for a similar purpose with the stereoscopic video recording.
3.3.4.
Visualization of motion capture data for training purposes
The motion analysis function of the motion capture applications are used as visualization tools for the purpose of visualization of motion capture data previously recorded. The data manager software allows downloading the motion files from the repository to the local machine running the motion capture data. The motion files can be open and reviewed using these applications. In addition the data manager software also allows downloading the raw videos and/or the annotated videos for review.
3.3.5.
Implementation of VRPN server
In the 2012 open call of the EXPERIMEDIA project two experiments were approved to be developed in the CAR Venue: CONFetti conducted by PSNC and 3D Acrobatic Sport developed by STT. In the kick-off meeting of these experiments it was seen that there were some areas of joint interest for both experiments. Therefore it was agreed to establish collaboration between those experiments in those activities where there is a given degree of overlapping. It was identified an area of collaboration consisting in the combination of the stereoscopic video recorded by the PSNC cameras with the three-dimensional motion capture data captured by the inertial sensors. The data obtained with the motion capture system can be shared with the software components developed within the CONFetti experiment and the combined result can be displayed on the computer screen. The practical implementation of this collaboration required sharing the data collected by the motion capture applications as well as the three-dimensional model representing the different
EXPERIMEDIA
elements of the skeleton. In order to implement this collaboration it was agreed upon the development of a VRPN server. The Virtual-Reality Peripheral Network (VRPN) is a set of classes within a library and a set of servers that are designed to implement a network-transparent interface between application programs and the set of physical devices (tracker, etc.) used in a virtual-reality (VR) system. The idea is to have a computer or other host at each virtual reality station that controls the peripherals (tracker, button device, haptic device, analogue inputs, sound, motion capture system, etc.). VRPN provides connections between the application and all of the devices using the appropriate class-of-service for each type of device sharing this link. The application remains unaware of the network topology. Note that it is possible to use VRPN with devices that are directly connected to the machine that the application is running on, either using separate control programs or running all as a single program. VRPN also provides an abstraction layer that makes all devices of the same base class look the same. This merely means that all trackers produce the same types of reports. At the same time, it is possible for an application that requires access to specialized features of a certain tracking device (for example, telling a certain type of tracker how often to generate reports), to derive a class that communicates with this type of tracker. If this specialized class were used with a tracker that did not understand how to set its update rate, the specialized commands would be ignored by that tracker. Each of these abstracts a set of semantics for a certain type of device. There are one or more servers for each type of device, and a client-side class to read values from the device and control its operation. Within the 3D Acrobatics experiment the implementation of the VRPN server for both the inertial and the optical motion capture applications was accomplished. This software was made available to PSC for integration in CONFetti experiment. The 3D models corresponding to the skeletons were delivered to PSNC.
3.4.
Equipment
Inertial motion capture system o o o o Set of 6 inertial sensors running at 250 Hz. Bluetooth antenna Tablet Laptop
During the experiment runs carried out at CAR Venue the following equipment was used.
15
EXPERIMEDIA
Figure 9. Motion capture system with inertial sensors.
Optical motion capture system o o o o o Set of 6 infrared cameras with resolution 832x832 pixels running at 250 Hz Set of 6 tripods and 6 camera supports Set of camera cables Ethernet switch PoE Laptop
Figure 10. Motion capture system with inertial sensors.
3.5.
Ethics and privacy
The 3D Acrobatics experiment recorded actual motion data from real athletes in the CAR Venue. Although the data recorded by the motion capture systems do not include any kind of picture or video it can be understood that a given person can be identified by her/his motion characteristics. This situation was discussed during the EXPERIMEDIA General Assembly held in Madrid in January 2013. It was agreed that the current Data Protection Act (DPA) used at CAR was valid for the development of the 3D Acrobatics experiment.
EXPERIMEDIA
After the meeting in Madrid a questionnaire was distributed by the partner responsible for Privacy Impact Assessment in the EXPERIMEDIA project including some questions about the Personal Data Flows regarding the experiments at CAR. This questionnaire was filled and submitted by CAR as well as the experimenters. As a result the partner responsible for PIA proposed an action plan composed of several points. The most relevant suggestion related to 3D Acrobatics experiment was the proposal to sign a controller-processor contract between CAR and STT.
17
EXPERIMEDIA
4. Experiment execution
Three different runs of the experiments were performed at CAR Venue in the following dates: First experiment run: May 15-16, 2013, Second experiment run: July 24th, 2013 and Third experiment run: September 16-18, 2013
4.1.
First experiment run
The first experiment run made use of the protocols implemented for the pommel horse. Previous tests were carried out at the biomechanical laboratory available in CAR premises. The results obtained in these early tests were satisfactory. After the previous tests, the experiment run with athletes was carried out in the main gymnasium available at CAR Venue premises. The run was executed in parallel with CONFetti experiment. The execution of this run revealed two different problems with the inertial motion capture system. On one hand the inertial sensors experienced problems to be connected to the host computer using Bluetooth protocol; on the other hand, the relative angles measured at the hips of the athlete showed a lack of accuracy. In order to identify the source of the connectivity problems new experiments were run a few days after by CAR staff. In these tests no electronic equipment different from the inertial sensors was present in the gymnasium. The output of these experiments revealed no problems with the sensor connectivity to the host computer if no additional electronic equipment was present in the gymnasium. The conclusion was that no corrective action should have to be undertaken.
Figure 11. Training session in pommel horse; motion capture data and stereoscopic video recorded simultaneously.
The lack of accuracy was examined in detail. It was concluded that the Kalman filter required more precise data from the triaxial gyroscope. The corrective action proposed was the improvement of the calibration procedure applied to the gyroscopes. This action required the re Copyright STT Ingeniera y Sistemas and other members of the EXPERIMEDIA consortium 2013 18
EXPERIMEDIA
calibration of the available sensors as well as a change in the firmware of the device. This change in the firmware required in turn a modification in the inertial motion capture software. Despite the two problems identified during this run of the experiment valid motion capture data were recorded during the training session. This data was recorded simultaneously with the stereoscopic video of the CONFetti experiment. As a result some augmented reality videos were created combining the images recorded by the stereoscopic cameras and the motion capture data streamed by the VRPN server. Figure 11 shows an athlete performing his exercises in the pommel horse and wearing a set of four inertial sensors. The purpose of these sensors was the measurement of the relative angles in the hip joints and lumbar spine. Figure 11 depicts a frame of an augmented reality content generated with the stereoscopic cameras by PSCN and the motion capture data collected by STT's sensors. The clip with the augmented reality contents was generated by PSCN combining the images recorded with the 3D cameras and the 3D model streamed using the VRPN server. The athlete participating in this run did not find any problem while wearing the inertial sensors. These sensors did not generate any problem in performing his exercises. He expressed that he was feeling comfortable wearing the sensors.
4.2.
Second experiment run
The second run of the experiment was conducted by STT technicians and CAR technical staff. This experiment was carried out using a new set of inertial sensors which incorporated the improved firmware for gyroscope calibration. All the tests in this run were carried out in the gymnasium available at CAR premises. The experiment was performed in three steps. The first set of tests was conducted with the purpose of verifying the connectivity between the inertial sensors and the host computer. Different connectivity tests were carried out connecting the sensors and transmitting data from different places of the gymnasium. No connectivity problems were detected even if the sensors were placed at up to 40 meters from the host computer.
Figure 12. Position of the inertial sensors for jump analysis.
The second set of tests focussed on the measurement of knee flexion motions and jumps. For this purpose two different athletes were appointed. Four inertial sensors were used placed in femur and tibia in both right and left legs simultaneously (see Figure 12). Different tests were performed with the two athletes. The results obtained in all the tests were very precise and
EXPERIMEDIA
provided the expected feedback. Athletes were satisfied with the results and expressed no objection in the use of the inertial sensors. Figure 13 shows the results obtained in one of these experiments; on the left side a skeleton showing the bones in the legs are depicted; on the right a graph with the flexion/extension angles in the knees is shown.
Figure 13. Results of jump analysis test.
Figure 14. Results of flexion/extension analysis test.
The third and final set of tests was performed on the same two athletes on the pommel horse. In this case the motion capture data obtained from the inertial sensors was not accurate. No connectivity problems were experienced but the accuracy of the relative angles in the hip joins was poor once the exercise started. The environment was examined and any potential source of magnetic distortion was discarded. As a consequence of these results a detailed revision of the theory, algorithms and implementation of the firmware and software in the inertial sensors was accomplished. Different experiments were designed in order to identify the source of poor accuracy in the motion
EXPERIMEDIA
performed by an athlete in the pommel horse. The results of this research revealed limitations in the implementation of the Kalman filter which computes the orientation of the inertial sensor. It was concluded that this control algorithm fails if the motion to be measured contains a component of permanent rotation around a given axis. In order to validate this conclusion all the experiments were performed with inertial sensors manufactured by different vendors. In both cases similar results were obtained.
Figure 15. Accuracy of relative angles measured with inertial sensors by different manufacturers.
Figure 15 shows the results obtained in one experiment performed with inertial sensors by different manufacturers. In this experiment three inertial sensors were located on a rigid body which rotated around the vertical axis with an angular speed of 360/s. This is approximately the same angular speed of an athlete working on the pommel horse. The graphs in Figure 15 depict the relative angle between two consecutive sensors in the rigid body. According to the conditions of the experiment this relative angle must be kept constant. However it can be noticed a different of about 30 in the left graph (which corresponds to STT sensor) and a difference of about 80 in the right graph (which corresponds to an inertial sensor by a manufacturer different from STT). The lack of accuracy of the motion capture system with inertial sensors for the pommel horse exercises forced the need of using an optical motion capture system for this type of exercises. The optical motion capture system did not impose any constraint neither to the athletes nor to the trainers. The optical markers are lighter and smaller than inertial sensors; they are not intrusive (as the inertial sensors) and allow the athletes to perform the exercises without limitation. The main drawback derived from this decision was the need to replicate in the software of the optical motion capture system the modifications that were already implemented in the software of the inertial motion capture system. This decision was taken by the end of August.
4.3.
Third experiment run
The third run of the 3D Acrobatics experiment made use of an optical motion capture system in order to overcome the limitations identified in the other two runs with the inertial sensors. During the third run several training sessions of pommel horse exercises were successfully recorded with two different athletes. Figure 16 and Figure 17 show side by side pictures with the athlete performing the exercise and the 3D avatar rendered in the computer screen by the optical
EXPERIMEDIA
motion capture application. These training sessions were recorded in two consecutive days and two different athletes participated in the experiment.
Figure 16. Training session in the pommel horse: circles and flairs.
Figure 17. Training session in the pommel horse: swings and scissors.
Figure 18. Training session in cycling.
The third day was devoted to recording training sessions in cycling. The purpose of this experiment was to record simultaneously the motion capture system with the Kinect cycling experiment driven by CERTH in the CAR Venue.
EXPERIMEDIA
In addition some of the training sessions in gymnastics and cycling were simultaneously recorded using the stereoscopic HD cameras in the CONFetti experiment. The collaboration with the CONFetti experiment is materialized by the implementation of a VRPN server as well as the sharing of the 3D model to be used in the combination of the results. This VRPN server was made available for both the inertial and optical motion capture solutions. This collaboration allowed to create augmented reality contents by combining the video recorded in the CONFetti experiment with the 3D models and motion data recorded using the motion capture systems.
Figure 19. Collaboration between three experiments within EXPERIMEDIA.
Figure 19 shows the teams of three different experiments working together in the same environment during the third run of the 3D Acrobatics experiment. The experience of the athletes using the optical motion capture system was very positive. The use of the optical markers did not impose any restriction to their motions during the exercises. The output generated by the motion capture software provided useful insight about the execution of the exercises being a value tool for improving the athlete performance.
23
EXPERIMEDIA
5. Results
5.1. Motion capture experience
The use of motion capture systems in real training sessions has been very positive. Both trainers and athletes have adopted the technology without any problem. Athletes felt comfortable using the motion capture devices (either inertial sensors or optical markers). The use of those devices did not impose any restriction during the execution of the exercises. The main goal of trainers and athletes is to improve the athlete's technique in order to achieve the optimal preparation for the competition. During the execution of the 3D Acrobatics experiment was found that this community is very open minded and are ready to adopt new elements which can potential help them to achieve a significant improvement in performance. Trainers and athletes are not particularly interested in the principles or the operation of the technological assets that are made available by the training centre. Instead they are more concerned about the practical results that can be achieved by the use of the technology. Keeping these concepts in mind the main goals derived from the use of the motion capture systems within the 3D Acrobatics experiment are described in the following points: It has been proven that the motion capture devices (inertial sensors or optical markers) are comfortably used by the athlete performing a training session. It has been verified that the results delivered by the motion capture systems provide additional value to the current training methods; trainers and athletes corroborated that the data provided by the motion capture systems are reliable and useful for improving the athlete's technique and the trainer's approach to training. It has been learnt that an outstanding value provided by the use of motion capture systems is the fact the evaluation is done in an "objective way" thanks to the quantitative values provided by the motion analysis software. It has been confirmed that the use of 3D avatars gives a better understanding of the exercise as well as the perception of the own body and motion; a more detailed analysis can give a better insight about the potential improvements than athletes can made in proprioception by using this kind of technologies. It has been shown that the athlete's understanding about her/his own motion and technique is significantly increased.
The number of training sessions carried out during the execution of the 3D Acrobatics experiment has been reduced. Therefore it has not been possible to verify that the use of motion capture systems significantly reduces the time required to improve the athlete's behaviour with respect to a given exercise or technique. In term of testing FMI products and services the experiment has totally reached the expected results. Nevertheless as the number of athletes using this system has been reduced the potential impact of this technology in FMI has only been tested in a reduced scope. It would be advisable to select a bigger group of users so as to cover a few tens of athletes in order to better evaluate the impact of FMI products and services.
EXPERIMEDIA
The overall evaluation of the results as well as the interviews with athletes, trainers and technical staff of CAR is that the tools used in the 3D Acrobatics experiment are a valuable tool for training and performance improvement in sports.
5.2.
Lessons learnt from FMI
The experimenters were interested in the development of the 3D Acrobatics experiment in order to look into the application of inertial motion capture systems in the training methods of high performance athletes as well as to explore the possibilities of FMI in sports training. FMI products and services are seen as a tool to reach a large community of users who are sport fans but not necessarily high performance athletes. The following list summarizes the most relevant lessons learnt by the experimenters: Inertial sensors are practical training sessions. Reliability of the motion capture data recorded in a practical training session has to be improved. Inertial sensors can be used under certain circumstances and for some exercises (for instance jumping or physical evaluation). Inertial as well as optical motion capture systems can be successfully applied in the training facilities of a high performance centre. The results provided by motion capture systems are relevant for both the trainer and the athlete for the improvement of the athlete's individual technique. Time required for the application of the motion capture results for the improvement of a practical training session is totally acceptable. It is very helpful to have the option of sharing the motion capture data, video and metadata through FMI products and services in order to improve athlete's individual technique.
During the execution of the 3D Acrobatics experiment the use of FMI products and services has been limited to sharing motion data files as well as video files. In addition although the size of the observation group was adequate for the scope of the analysis proposed in the experiment it does not reveals trends in the potential impact of the used tools in a larger social network. From the perspective of the FMI functions it would be advisable to enlarge the scope of the experiment in order to encompass a larger community of athletes and trainers. In order to accomplish this goal it would be recommended to select another type of sports speciality such as soccer, basketball or handball. These sports are extremely popular in most of European countries and the number of sport schools managing young teams has a tremendous potential to perform the analysis outlined in this paragraph. For this application the results of the analysis tools should be organized in a different way. A suitable approach would be creating a social network tool for athletes, trainers and sports enthusiasts. Following these suggestions it would be possible to carry out a more detailed exam regarding the potential impact that objective physical evaluation and training programs based on the use of motion capture systems can have in today's society.
25
EXPERIMEDIA
5.3.
Dissemination
STT has devoted efforts to the dissemination of the 3D Acrobatics results among its commercial partners and customers. To this end a logo was designed in order to provide a visual image of the new services and products (see Figure 20).
Figure 20 3D Acrobatics logo
As a result of these dissemination activities, the following entities have showed interest in the outcome of the 3D Acrobatics experiment: Motion Sports Institute (http://www.motionsportsinstitute.com/SitePages/Home.aspx). This company develops a product called web-locker which is aimed at sharing useful information for athletes over Internet. The FMI products and services in combination with the motion capture systems for objective physical evaluation has attracted the attention of this company. Atlantic Health (http://www.atlantichealth.org). This company is a potential partner of STT in USA which is looking an expanding its activities in the field of physical evaluation on sports and rehabilitation. The FMI tools would allow sharing the information between health professionals and patients/athletes in a very efficient way. Indiana University (http://www.indiana.edu). STT introduced the results of 3D Acrobatics experiment to managers of Indiana University. This university has established a strong partnership program in sports with colleges and high schools. The availability of tools such those implemented in this experiment are of tremendous interest in a partnership program like the one implemented by Indiana University.
26
EXPERIMEDIA
6. Conclusions
The execution of the 3D Acrobatics experiment led to the conclusion that the proposed technology provides a tremendous added value to the sports community. The combination of motion capture systems with FMI products and services represents an important contribution to the sports community in the area of sports training and performance improvement. The use of motion capture technologies has proven to provide a useful feedback both to athletes and trainers. It gives an instantaneous view of the exercise performed by the athlete offering a better understanding of the training and execution techniques. The runs of the experiment also highlighted some limitations on the use of inertial sensors for certain types of evaluation. It has been proven that the inertial motion capture technology can be successfully applied in some exercises. This fact together with the portability of the system (can be used outdoors, can be used in the field, there are no limitations imposed by the environment, etc.) makes this technology an interesting alternative for carrying out analysis in the field. This is an affordable that can be used not only with professional teams but also with amateur teams, schools, etc. Considering the conclusions extracted of the experiment execution, STT is considering three different action lines to continue with the activities started with the 3D Acrobatics experiment. These lines are organized in different levels: technical, evaluation and verification and exploitation. At the technical level, STT considers the improvement of the algorithms and methods implemented for the fusion algorithm. This development will enlarge the area of potential application of the technology used in the 3D Acrobatics experiment. At the evaluation level, STT is aiming at continuing the collaboration with CAR in the application of the inertial sensors in the analysis and evaluation of knees in soccer teams. This area has been identified as an application area in which this technology may have a great impact. Finally, at exploitation level, STT will pursue the dissemination activities started with its partners and customers in USA.
27

D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0 PDF

Uploaded by

Copyright:

Available Formats

D4.7.

3 Experiment Results and Recommendation

Experiment execution ...................................................................................................................... 18

Figure 1. Experiment runs at CAR.

3. Description of the Experiment

Figure 2. Inertial sensor STT-IBS.

Figure 3. Inertial sensors attached to the athlete's body.

Overview of the Experiment Design

Figure 4. System architecture.

Components Developed within the 3D Acrobatics Experiment

EXPERIMEDIA Data collection

Figure 6. Two positions during pommel horse exercise.

Figure 7. Optical marker set for cycling analysis.

Generation of video contents

Synchronization with metadata

Figure 8. Application for synchronization of video and metadata.

Visualization of motion capture data for training purposes

Implementation of VRPN server

Figure 9. Motion capture system with inertial sensors.

Figure 10. Motion capture system with inertial sensors.

Ethics and privacy

First experiment run

Second experiment run

Figure 12. Position of the inertial sensors for jump analysis.

Figure 13. Results of jump analysis test.

Figure 14. Results of flexion/extension analysis test.

Third experiment run

Figure 18. Training session in cycling.

Figure 19. Collaboration between three experiments within EXPERIMEDIA.

Lessons learnt from FMI

Figure 20 3D Acrobatics logo

You might also like