Professional Documents
Culture Documents
Multivariate Workload Evaluation Combining Physiological and Subjective Measures
Multivariate Workload Evaluation Combining Physiological and Subjective Measures
Multivariate Workload Evaluation Combining Physiological and Subjective Measures
233᎐238
Shinji MiyakeU
Department of En¨ ironmental Management II, School of Health Sciences, Uni¨ ersity of Occupational and En¨ ironmental
Health, Japan 1-1 Iseigaoka, Yahatanishiku, Kitakyushu 807-8555, Japan
Received 6 January 2000; received in revised form 6 June 2000; accepted 15 June 2000
Abstract
This paper suggests a way to integrate different parameters into one index and results obtained by a newly
developed index. The multivariate workload evaluation index, which integrates physiological parameters and one
subjective parameter through Principle Components Analysis, was proposed to characterize task specific responses
and individual differences in response patterns to mental tasks. Three different types of mental tasks were performed
by 12 male participants. Heart rate variability, finger plethysmogram amplitude, and perspiration were used as
physiological parameters. Three subscales, mental demand, temporal demand and effort out of six subscales in the
NASA-Task Load Index were used as subjective scores. These parameters were standardized within each participant
and then combined. It was possible to assess workload using this method from two different aspects, i.e. physiological
and subjective, simultaneously. 䊚 2001 Elsevier Science B.V. All rights reserved.
Keywords: Principal components analysis; HRV; Plethysmogram; Perspiration; NASA-TLX; Mental workload; Subjective
0167-8760r01r$ - see front matter 䊚 2001 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 7 - 8 7 6 0 Ž 0 0 . 0 0 1 9 1 - 4
234 S. Miyake r International Journal of Psychophysiology 40 (2001) 233᎐238
workload effects. On the other hand, the re- 1.4 pixelrunit time, respectively. and the width of
sponse sensitivity to a mental task is different in the target moving area Ž350 = 330, 167 = 166 and
each person. The physiological responses induced 86 = 83 pixels for H, M and L, respectively.. The
by the same task may also differ from person to tracking task was similar to a simple reaction time
person. This is the individual difference problem task, however, some precognition Žprediction. may
ŽTurner, 1994.. Furthermore, the physiological re- have been necessary. The logical task was nearly
sponse pattern is different from task to task. For identical to the Mine Sweeper game in Microsoft
example, the response induced by the mental Windows ŽR.. In this task, the participants were
arithmetic task is different from the response told to guess whether there was a mine in a grid
induced by the mirror drawing task ŽSaab and cell and to click the ‘safe’ cells. In this task, even
Schneiderman 1993; Miyake 1997.. Thus, we must if the participants hit the mine, the task was not
consider such individual differences and task spe- finished as in the Windows game. The more dif-
cific response patterns when workload research is ficult the level, the more mines there were. This
investigated. One approach to solving these prob-
logical task may have required short-term me-
lems is to record and analyze several physiological
mory and logical inference. The task duration was
Žand subjective. responses with different at-
4 min for each difficulty level in the T and L
tributes and integrate them in a way which can
tasks. The tasks were ordered from more difficult
reflect individual differences in physiological sen-
to easier and the same fixed order was applied for
sitivity and task specific responses. The purpose
all participants, i.e. P, TH , L H , TM , L M , TL and
of this study was to investigate a new method of
mental workload assessment in which multiple L L ŽMiyake, 1996.. Using this approach, the task
physiological parameters and subjective indexes difficulty factor nullified the training effect and
are integrated into one index through multivari- emphasized the differences in difficulty levels
ate analysis. among tasks. The computer tasks ŽT and L. were
programmed by QuickBASIC 4.5 ŽMicrosoft. and
run on an MS-DOS operated 486 computer with
a 15-inch Ž640 = 400 pixels. CRT display.
2. Method Participants were 12 male university students
ranging in age from 18.9 to 25.9 years with an
average age of 22.7 years. All participants gave
Three different kinds of mental tasks were used: their informed consent before the experiments
a six-piece wooden puzzle ŽP., a two-dimensional and were given the same amount of payment
compensatory tracking task with a first-order con- regardless of their task performances.
trol ŽT., and a numerical logical task ŽL.. The T Three different physiological measures were
task and the L task had three levels of difficulty, acquired during all experimental blocks, including
high ŽH., medium ŽM. and low ŽL.. Therefore, before ŽPRE. and after ŽPOST. task rest periods
they were abbreviated as TH , TM and TL for the Ž5 min.: Ž1. the ln LFrHF Žnatural logarithm of
tracking task and L H , L M and L L for the logical
LF to HF ratio 1 . as a cardiovascular parameter;
task. In the P task, participants were instructed to Ž2. the photoelectric plethysmogram amplitude
make a simple silhouette pattern Žcross. using all
six wooden pieces ŽCross Puzzle II, D.1 Products.
in 8 min. This task requires the pattern recogni-
tion ability. The T task required participants to
1
keep an airplane icon target inside a central LF is the low-frequency component of HRV of approxi-
circular gunsight area using a joystick controller mately 0.10 Hz that primarily reflects baroreceptor-mediated
ŽFlight Stick Pro, CH Products.. The task dif- regulation of blood pressure. HF is the relatively high-
frequency component of approximately 0.25 Hz that corre-
ficulty level was controlled by the target speed sponds to the frequency of respiration in HRV spectral com-
Žaverage speed for H, M and L were 8.0, 3.4 and ponents.
S. Miyake r International Journal of Psychophysiology 40 (2001) 233᎐238 235
measured on the left index finger as a peripheral was sampled at 100 Hz, and the average ampli-
blood vessel activity parameter; and Ž3. the tude of this signal was obtained as an index for
amount of perspiration from a sweat rate meter the amount of perspiration. The subjective work-
attached to the left thumb as a non-vascular load score was obtained by using the NASA Task
autonomic nervous system parameter. The partic- Load Index ŽTLX. ŽHart and Staveland, 1988.
ipants were instructed to synchronize their respi- which contains six subscales: Mental Demand
ration pace with a computer generated tone dur- ŽMD., Physical Demand ŽPD., Temporal Demand
ing the rest and task periods to reduce the effect ŽTD., Own Performance ŽOP., Effort ŽEF. and
of irregular respiration on the HRV power spec- Frustration level ŽFR.. The NASA-TLX rating
tral components ŽGrossman et al., 1991; Hayano window automatically appeared on the computer
et al., 1994.. The respiration signal was recorded screen when each task duration had expired. Then
by a strain-gauge around the chest to monitor the the participants used the mouse cursor to rate
respiration regularity and to identify the respira- their subjective workload.
tory sinusarrythmia component in the HRV power Two participants were rejected from subse-
spectrum. quent analyses. A clear HF component was not
The HRV spectral analysis and other physio- detected in one participant because his respira-
logical data analyses were done as follows. A 30-s tion rate was so slow in one block that the HF
duration from the beginning of each block was component was contaminated by the LF compo-
not included in these analyses to reject the tran- nent. The other participant showed highly irregu-
sient, relatively large drift in signals which is lar respiration in one block even though he was
frequently observed just after the task started. A instructed to control his breath as described
200-s ECG recorded from the CM 5 lead ŽElles- above. Therefore, no clear HF component was
tad, 1986. was sampled at 1 kHz in each block. found in his HRV power spectrum.
The near-DC components Ž- 0.05 Hz. in the R-R All parameters were standardized within partic-
interval were removed by a digital filter, and the ipants. Physiological workload evaluation ŽPWE.
equidistant R-R interval data were obtained by scores were calculated by means of the PCA of
resampling Ž2 Hz. a spline-interpolated trend- the three physiological parameters mentioned
gram. The 10th-order AR spectral analysis was above. The Weighted Workload ŽWWL. score
applied ŽMiyake et al., 1994.. The LF and HF and the average score of MD, TD and EF sub-
components were extracted by the spectral de- scales of NASA-TLX, which was labeled as the
composition method, and the ln LFrHF was se- TLX-MTE ŽMental, Temporal and Effort. score,
lected for the Principal Components Analysis were calculated. This TLX-MTE score was de-
ŽPCA. procedure. A 200-s plethysmogram signed according to the Subjective Workload As-
recorded by a photoelectric plethysmograph sessment Technique ŽReid and Nygren, 1988.
ŽNihon-Koden MLV-2301. was sampled at 1 kHz, which contains only three dimensions, i.e. Time
and 10-ms interval data were obtained by resam- Load, Mental Effort Load and Psychological
pling every other tenth point Žcoarse graining.. Stress Load. The Time Load seems to be identical
Baseline fluctuation was removed by a 100-point to TD, and Mental Effort Load is similar with
moving average Žequivalent to a 0.443-Hz high MD and EF. The Psychological Stress Load may
pass filter., and the root mean squared ŽRMS. be equivalent to the FR in NASA-TLX. However,
value was calculated as an average beat compo- FR was not included in the TLX-MTE score. FR
nent amplitude. A small capsule with a highly was excluded because the purpose of this new
accurate static capacity moisture sensor and a subjective scale was to reflect subjective feelings
temperature sensor was attached on the skin sur- during the task; and it was assumed that the
face of the left thumb, and perspiration signal was frustration level ŽFR. rated after the task might
measured by the direct capsule method ŽSuzuken be affected greatly by the task result, i.e. success
Kenz-Perspiro OSS-100.. The perspiration signal or failure.
236 S. Miyake r International Journal of Psychophysiology 40 (2001) 233᎐238
MWE jk s W1 k P1 jk q W2 k P2 jk q W3 k P3 jk
q W4 k S1 jk Ž1.