Journal of Strength and Conditioning Research Publish Ahead of Print

DOI: 10.1519/JSC.0000000000001894

U.S. Army Physical Demands Study: Reliability of Simulations of Physically

Demanding Tasks Performed by Combat Arms Soldiers

D
Stephen A. Foulis, Jan E. Redmond, Peter N. Frykman, MAJ Bradley J. Warr, Edward

J. Zambraski, Marilyn A. Sharp

TE
Military Performance Division, U.S. Army Research Institute of Environmental Medicine,

Natick, MA, USA

EP
Running Head: Reliability of Simulations

Corresponding Author:
C

Stephen Foulis

10 General Greene Ave, BLDG 42

C

Natick, MA 01760

Phone: 508-233-4884
A

Fax: 508-233-4195

Email: stephen.a.foulis.civ@mail.mil

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 1

ABSTRACT

Recently, the US Army has mandated that Soldiers must successfully complete the

physically demanding tasks of their job in order to graduate from their Initial Military

Training. Evaluating individual Soldiers in the field is difficult; however, simulations of

these tasks may aid in the assessment of Soldiers’ abilities. The purpose of this study

D
was to determine the reliability of simulated physical Soldiering tasks relevant to

Combat Arms Soldiers. Three cohorts of ~50 Soldiers repeated a subset of eight

TE
simulated tasks four times over two weeks. Simulations included: sandbag carry,

casualty drag, and casualty evacuation from a vehicle turret, move under direct fire,

stow ammunition on a tank, load the main gun of a tank, transferring ammunition with a

field artillery supply vehicle, and a 4-mile foot march. Reliability was assessed using
EP
intraclass correlations (ICC), standard errors of the measure (SEM), and 95% limits of

agreement (LOAs). Performance of the casualty drag and foot march did not improve

across trials (p>0.05) while improvements, suggestive of learning effects, were

C

observed on the remaining six tasks (p≤0.05). The ICCs ranged from 0.76-0.96, and

the SEMs ranged from 3%-16% of the mean. These eight simulated tasks show high
C

reliability. Given proper practice, they are suitable for evaluating the ability of Combat

Arms Soldiers to complete the physical requirements of their jobs.

A

Key Words: Job Task; Learning Effect; Repeatability

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 2

INTRODUCTION

Performing physically demanding tasks is an integral part of being a Soldier in

the U.S. Army (15). This is particularly true of the Combat Arms military occupational

specialties (MOS), to include Infantry, Armor, Combat Engineers, and Field Artillery.

Physical ability in the U.S. Army is typically measured using the Army Physical Fitness

D
Test (APFT), which includes sit-ups, push-ups, and a 2-mile run (18). Scores on the

APFT events have a poor relationship with a Soldier’s ability to perform Soldiering tasks

TE
(3, 4). The APFT was designed as a measure of health and fitness, not as a measure

of job performance (8).

Recently, the US Army has mandated that Soldiers must successfully complete
EP
the physically demanding tasks of their MOS in order to graduate from their initial

military training. The intent in this requirement is to improve Soldier readiness and

reduce the incidence of musculoskeletal injuries. The ideal method for the Army, or any
C

tactical strength and conditioning (TSAC) practitioner, to assess physical performance

would be to actually perform the task; however, this is difficult or impossible in many
C

cases. Many tasks require multiple people, resulting in the inability to determine the

success of the individual. Some tasks, such as loading and firing a tank round, may be
A

expensive to perform or require hard-to-obtain equipment. Safety can also be an issue,

especially during prolonged tasks with heavy loads. Finally, field tests can be unreliable

due to technical and site conditions (17). The use of task simulations can overcome

many of these obstacles, while potentially maintaining a strong relationship with the

original job task (6, 11).

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 3

Other militaries have developed task simulations to aid in the evaluation of their

Soldiers. The Australian Defence Force has simulated Soldier performance on strength

endurance tasks, such as reloading equipment or carrying a casualty, by completing a

simulated jerry can carry (12). To determine the ability to lift heavy objects, such as

D
installing equipment or lifting a casualty, the British Army has used a maximal box lift

(13, 14). For the Canadian Armed Forces, the task definition of a land evacuation

TE
required two people to carry a stretcher (17). In order to test individuals, they

developed a simulation where a Soldier would carry half the load on a stretcher with

wheels on one side.

EP
From 2013-2015, the U.S. Army conducted the “Physical Demands Study” of

Combat Arms Soldiers to develop a pre-enlistment screening test to predict success on

the physically demanding tasks performed by these Soldiers (5). As a corollary to this
C

study, eight task simulations of the most physically demanding tasks were developed:

sandbag carry, casualty drag, casualty evacuation from a vehicle turret, move under
C

direct fire, stow ammo on a tank, load the main gun of a tank, and transfer ammunition

with a field artillery ammunition supply vehicle (FAASV). The Army has begun using
A

these task simulations at the end of initial entry training to determine if an individual can

or cannot complete these tasks to minimal required standard of their MOS.

Furthermore, some of these task simulations could provide useful evaluation tools for

measuring performance of Soldiers and other similar tactical athletes.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 4

In order for a task simulation to be useful as an evaluation tool for Army or other

TSAC professionals, the task simulations need to be reliable. The same person must

get the same score every time they test. Other researchers have conducted reliability

analyses on military relevant job task simulations (10, 16); however, the eight task

simulations developed in the Physical Demands Study have not been evaluated. The

D
purpose of the present study was to determine the learning effects and test-retest

reliability of simulations of these eight tasks.

TE
METHODS

Experimental Approach to the Problem

Data were collected from three cohorts of ~50 Soldiers. Each cohort completed a
EP
subset of the eight task simulations four times over the course of two weeks. For three

tasks, custom structures were developed. Schematics of these structures are

previously published (5). Outcome variables were selected to provide a range of

C

scores, and ensure all Soldiers who attempt the task received a score. Scores were

compared over time to test for learning effects and reliability. Learning effects are
C

defined as systematic, significant improvements in their score across consecutive trials.

Reliability was calculated using both relative (same rank within cohort compared to
A

previous attempt) and absolute (same score compared to previous attempt) metrics.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 5

Subjects

A total of 149 active duty Soldiers (79 men, 70 women; age 18-36 years) were

recruited for participation in this portion of the study. With the Army’s direct ground

combat exclusion lifted in 2013 allowing for the opening of all Combat MOSs to women,

it was important to capture a sample of both males and females. Each cohort consisted

D
of 49 or 50 Soldiers. Cohort 1 was tested in May 2014, while Cohorts 2 and 3 were

tested in December 2014. Although Soldiers were not required to be of an MOS that

TE
typically performed these tasks, 31 Soldiers were members of the Combat Arms.

The investigators have adhered to the policies for protection of human subjects

as prescribed in Army Regulation 70-25, and the research was conducted in adherence
EP
with the provisions of 45 Code of Federal Regulations (CFR) Part 46. All Soldiers were

briefed on methodology and risks of performing the tasks prior to signing an informed

consent document approved by the USARIEM Institutional Review Board. After

C

obtaining consent and completing screening forms, participating Soldiers were asked to

complete a demographics and training history questionnaire. Anthropometrics (height,

C

weight) were also collected prior to testing.

A

Soldier characteristics are provided in Table 1. Soldiers were in various points in

their training cycle and were not currently preparing for deployment. They were from 48

different MOSs, and there was a broad range of familiarity with the tasks prior to testing.

Of the Soldiers completing each task, 46% self-reported experience with a casualty

evacuation, 84% with carrying sandbags, 82% with a casualty drag, 90% with a move

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 6

under fire task, 2% with stowing ammo on an Abrams, 2% with loading the main gun of

an Abrams, 4% with transferring ammo on a FAASV, and 88% with completing a road

march of at least 4 miles.

***INSERT TABLE 1 HERE***

D
Procedures

TE
Each task was performed once per day at the same time of day over the course

of four testing days. At least 48 hours was provided between repeating tasks, and no

more than two trials were performed within 7 days. Soldiers were instructed to match

their routine (e.g. diet, sleep) as closely as possible prior to each session, with the
EP
exception that on testing days, they were instructed not to participate in morning

physical training. All aspects of the testing (instructions, uniform, etc.) were matched as

closely as possible at each testing session. Participating Soldiers were asked to

C

complete a familiarization or practice trial once prior to each testing session. They were

provided with a minimum of 10 minutes of rest between different job task simulations.
C

Sandbag Carry (Cohort 1)

A

Soldiers lifted and carried 16 sandbags weighing 18.1 kg while wearing a fighting

load without a weapon (approximately 32 kg). Sandbags were carried 10 meters and

placed on the floor in a 4 long x 2 wide x 2 high position as quickly as possible. Soldiers

were permitted to carry no more than two sandbags at a time. Time to complete the

task was collected (min).

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 7

Casualty Drag (Cohort 1)

Soldiers dragged a modified Survivor dummy (Dummies Unlimited, Pomona, CA;

approximately 123 kg, the weight of an average Soldier wearing a fighting load) a

distance of 15 m as quickly as possible on a polished concrete floor. Soldiers were

D
wearing a fighting load (approximately 38 kg). The dummy was outfitted with a modified

Fighting Load Carrier to serve as a pulling handle and weighed down to achieve 123 kg.

TE
Time to move the dummy 15 m was recorded. If the Soldier failed to pull the casualty

15 m in 30 s, the distance the casualty was dragged was measured. Scores were

calculated as the velocity (m·s-1) at which the dummy was moved.

EP
Casualty Evacuation from a Vehicle Turret (Cohort 1)

A custom-built platform was constructed with a cutout matching the dimensions

of the hatch of Bradley Fighting Vehicle (5). A heavy bag (Black Diamond Touchstone,
C

Salt Lake City, UT) was placed underneath the opening to simulate the casualty (Figure

1A). The heavy bag was modified to include straps that simulate the shoulder straps of
C

a Combat Vehicle Crewman protective vest. The bag was placed in the hole, with the

handles of the bag level with the platform. Soldiers performed this task wearing a
A

fighting load without a weapon (approximately 32 kg).

Prior to initiating the task, each Soldier practiced proper lifting technique using a

pair of 10 kg kettle bells. For the actual tests, once on the platform, Soldiers squatted

and grasped the handles of the heavy bag and then stood up and pulled the bag

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 8

through the hole in the platform. Soldiers were required to place the heavy bag onto the

platform for successful task completion. An initial load of 22.7 kg was used for additional

familiarization and warm-up. With the successful completion of each lift, the weight of

the simulated casualty was increased in 4.5-, 9.1-, or 13.6-kg increments. Following at

least 3 min of rest at the higher loads (>80% one repetition maximum), the process was

D
repeated until the Soldier reached volitional fatigue, showed bad form, failed to lift the

bag during 2 consecutive attempts, or reached a maximum load of 95.5 kg. The

TE
maximum load represented the weight of an average Soldier wearing a Vehicle

Crewman Uniform. If Soldiers were not able to lift the bag following an increment of

more than 9.1 kg, the Soldier was allowed to decrease the weight and try again. The

maximal load was recorded (kg).

EP
Move Under Direct Fire Simulation (Cohort 2)

During this task, Soldiers wore a fighting load (approximately 38 kg) and carried
C

a simulated weapon at the ready. Soldiers began the task in the prone position. Upon

command, Soldiers got up, sprinted approximately 6.6 meters to a marker, and
C

assumed the predetermined position for that marker (either the kneeling or prone

position). They remained in this position for 5 seconds. Upon signal, Soldiers got up and
A

sprinted to the next marker and assumed the predetermined position for that marker.

The order of the positions was kneel, kneel, prone. This was repeated until a total of

100 meters (15 movements) was completed. Time to complete the task was recorded

(min).

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 9

Stow Ammunition on Abrams Tank (Cohort 2)

For this task, Soldiers wore a fighting load without a weapon (approximately 32

kg). Soldiers carried and lifted eighteen 120mm multi-purpose anti-tank projectile

(MPAT) rounds (approximately 25 kg each) from a rack simulating an ammunition point

up to a 1.6m tall platform simulating the position of a person on deck of an Abrams tank

D
(Figure 1B) (5). This platform was 5 meters away the ammunition rack. An assistant

stood waiting at the platform to receive the round. Time to complete the task was

TE
recorded, and a rate (rounds·min-1) of loading the rounds was calculated. If Soldiers

were unable to complete the task within 15 min or chose to stop, the rate was calculated

as the number of rounds completed divided by 15 min.

EP
Load the 120mm Main Gun on an Abrams Tank (Cohort 2)

While wearing approximately 22 kg of task specific equipment, Soldiers loaded

five 120mm MPAT rounds (approximately 25 kg each) into a simulated breach of the
C

Abrams tank main gun as quickly as possible (Figure 1C). A wooden simulator

matching the dimensions of the inside of an M1 Abrams was constructed (5). Prior to
C

testing, Soldiers were briefed on proper technique. Soldiers completed two practice

trials at a submaximal pace and then three maximum trials for record. Time to complete
A

the task was recorded and the fastest two trials were averaged.

Transfer Ammunition with an M992 Field Artillery Ammunition Supply Vehicle (FAASV)

(Cohort 2)

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 10

While wearing approximately 22 kg of task specific equipment, each Soldier had

20 minutes to move thirty demilitarized M795 High Explosive Rounds (approximately 45

kg each). Rounds were moved from the floor at the tailgate of the FAASV into the

designated locations on the ammunition rack on the back of the vehicle. In accordance

with standard operating procedure, the highest point on the rack that the Soldier was

D
required to place the round was equal to the shoulder height of the Soldier. Soldiers

were given lower openings on the rack to make up for any that were above their

TE
shoulder height. The 20-minute task was divided into three 5-minute active loading

segments with two 2.5-minute rest periods. Time to complete the task (not including the

3-minute rest period) was recorded. If the Soldier was moving a round at the end of the

5-minute active period, he or she was asked to place it down safely. The next active
EP
period started from where the round was placed.

The rate of loading (rounds·min-1) was calculated either by dividing 30 rounds by

C

the elapsed time (if all rounds loaded were before the time expired) or by dividing the

number of completed rounds by 15 minutes (if all 30 rounds were not loaded).
C

4-Mile Foot March (Cohort 3)

A

The foot march simulation required Soldiers to complete a foot movement of

approximately 6.4 km (4 miles) on a relatively flat course with a firm surface (packed dirt

or macadam). Based off previous data (REF Paper 2) and subject matter expert input,

a 6.4 km march was sufficient to predict success on a 20 km march (the distance

required by the Army standard). Soldiers wore an Army Combat Uniform, personal

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 11

protective equipment (including weapon), and 24-hour sustainment load (approximately

47 kg total). Soldiers were instructed to complete the task as quickly as possible while

walking on a supervised course. Running and the ‘airborne shuffle’ were not allowed.

Soldiers were allowed to take breaks as needed. Soldiers were instrumented with a

timing chip (SPORTident Active Card SIAC, Arnstadt, Germany). Time to completion

D
was recorded.

TE
***INSERT FIGURE 1 HERE***

Statistical Analyses

The sample size was determined by using the sample size estimation formula of
EP
Hopkins (7) and data on repetitive lifting tasks from Pandorf (10), which indicated that

37 Soldiers would be needed to see a difference in scores at the p<0.05 level. All

statistics were calculated using SPSS Version 20 (IBM Corporation, Armonk, New
C

York). Significance was set at the p<0.05 level. Descriptive statistics were calculated to

characterize the population. Because our purpose was to understand the reliability of
C

the tasks, and we were underpowered to run any sex comparison, no separate sex

analyses were run.

A

The statistical approach was based on the method used by others in determining

the reliability of tasks (1, 16). Repeated measures analysis of variance (ANOVA) with a

Tukey’s post-hoc multiple comparison test was employed for each task to evaluate the

presence of a systemic bias (i.e. learning effect) between trials. Improvement between

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 12

successive trials was considered indicative of a learning effect. The first two trials

without a learning effect were used to determine reliability of the task.

Reliability coefficients and their associated 95% confidence intervals (95% CI)

were examined across trials to determine task reliability. Random error in the

D
measurements was assessed as relative reliability and absolute reliability (1). Relative

reliability was assessed with intraclass correlation coefficients (ICCs) while absolute

TE
reliability was assessed using Standard Error of Measurement (SEM) and 95% limits of

agreement (95% LOA). ICCs were calculated using a two-way random effects, single-

measure reliability model. SEMs are reported in both absolute units and as a

percentage of the mean. The 95% LOA were calculated as either the 95% ratio LOA if
EP
the error increased as the scores increased (as determined by a Bland-Altman plot (2)).

If the Bland-Altman plot indicated the test-retest error was constant, the 95% LOA was

calculated.
C

RESULTS
C

Marginal means and standard errors of the scores for each of the task

simulations during each test session are provided in Table 2. The average weight for all
A

trials for the casualty evacuation was 70.0 ± 10.0 kg; average time for the sandbag

carry was 2.10 ± 0.61 minutes, and average velocity for the casualty drag was 1.07 ±

0.36 m·s-1. Average time for the move under fire was 2.26 ± 0.23 min; average rate of

the stow ammo was 4.50 ± 2.89 rounds·min-1, average time to load the main gun was

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 13

26.97 ± 6.94 s, and the average rate for transfer ammo with a FAASV was 3.10 ± 1.67

rounds·min-1. The average time for the foot march was 80.29 ± 12.09 min.

There were no significant differences in individual Soldier’s scores across trials

for the casualty drag or the foot march (p≥0.10). Significant improvements in scores

D
were recorded during the second trials for the remaining 6 tasks (p≤0.05), indicating a

possible learning effect. There were no further improvements in the scores for the

TE
sandbag carry, casualty evacuation, or move under fire during the third trials (p≥0.15).

Significant improvements continued to be observed from the second to third trial for the

FAASV task (p<0.01), but not the final trial (p=0.09). Stow ammo and the main gun and

improved through all trials (p<0.01).

EP
***INSERT TABLE 2 HERE***
C

Reliability data are presented in Table 3. ICCs of the tasks ranged from 0.76

(95% CI: 0.61-0.86) for the foot march to 0.96 (95% CI: 0.94-0.98) for the casualty
C

evacuation. In terms of the absolute reliability, the SEMs ranged from 3% of the mean to

16% (18% if the learning effect is not accounted for). The 95% LOAs were 33% for the
A

sandbag carry time, 0.35 m·s-1 for the casualty drag, 11.7 kg for the casualty

evacuation, 0.16 minutes for the move under fire, 1.59 rounds·min-1 for the stow ammo,

21% for load the main gun, 1.25 round/minute for the FAASV and 16.34 minutes for the

foot march.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 14

***INSERT TABLE 3 HERE***

DISCUSSION

This study established the reliability of simulations of eight physically demanding

D
tasks of Combat Arms Soldiers.

TE
For the casualty drag and the foot march, there were no additional improvements

after the second trial. This indicates there were no learning effects on these tasks. It is

possible that the simulations have removed any trainable skills; however, it also notable

that Soldiers had the greatest prior experience with these tasks. These two tasks
EP
represent common Soldiering tasks that Soldiers are expected to be able to perform,

and the simulations for these two tasks were faithful to the original task required of the

job.
C

There is evidence of systematic bias for a number of the tasks, such that better
C

scores are observed on subsequent trials. The cause of this increase in performance is

likely due to a significant learning effect (Table 2) (1). Notably, the five tasks which
A

showed no learning effect (casualty drag, foot march), or no additional learning effect

after the second trial (sandbag carry, casualty evacuation, move under fire), were all

common Soldiering tasks. Most Soldiers are expected to be able to be perform these

tasks. While Soldiers likely had experience with the sandbag carry, casualty

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 15

evacuation, and move under fire tasks, it may have taken an additional trial familiarize

themselves with the way the simulations differed from their training.

The remaining three simulations (FAASV, stow ammo, loading the main gun) had

a longer lasting learning effect. These tasks were job-specific, and Soldiers reported

D
the least familiarity with these tasks. Stow ammo and load the main gun continued to

show improvements over all four sessions. Due to limited availability of the Soldiers, it

TE
was not possible to run any additional trials. Despite the continued improvement, the

stow ammo and main gun had ICCs ≥ 0.93, indicating a consistent improvement across

all participants. Between the third and fourth trial, Soldiers improved an average of 20 s

(10%) and 2 s (12%) on the stow ammo and main gun tasks, respectively. While
EP
additional trials may completely mitigate the learning effect, these error rates may be

acceptable for assessing physical demands of these tasks.

C

Prior to testing all tasks, Soldiers were given a brief familiarization and practice.

This training was only to teach Soldiers any skill and technique required. Before being
C

used for technical purposes, practice is essential for mitigating any learning effects,

particularly the FAASV, stow ammo, and loading the main gun.
A

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 16

Reliability

We used three measures of reliability in this study: ICC, SEM, and 95% LOA.

The ICC is an indicator of relative reliability. High ICCs are indicative of a test that is

able to consistently rank participants, independent of actual score (i.e., the order of

completing the task relative to their peers). As such, all of the criterion tasks had ICCs

D
with upper bounds of their 95% CI >0.80. The task with the lowest ICC (0.76 (95% CI:

0.61-0.86)) was the foot march. There is no formal consensus of a minimal acceptable

TE
level of reliability for a performance test. Literature values suggest that an ICC > 0.75 is

considered acceptable for clinical research (19); however, the authors are unaware of

any legally acceptable standard.

EP
Two measures of absolute reliability (SEM & 95% LOA, Table 3) provide an

indication of the variability between repeated trials, independent of participants

performance ranking in the sample. The SEM describes the general variability of the
C

sample around its true value. While it is traditionally used as a reliability measure, it is

difficult to interpret this value’s meaning on the reliability of an individual’s score. This
C

limits its usefulness in delineating specific cut-offs of what is acceptable reliability. A

separate value is the 95% LOA, which treats the data as a population of test-retest
A

differences (1) and calculates test-retest differences for 95% of the population. Absolute

LOA are used when there is be uniform error across all scores (e.g., ± 1 kg for both a

score of 10 and a score of 20 kg), while Ratio LOA is used when the results indicate

individuals with a higher score would have greater error (e.g., ± 10% of the score: ± 1 kg

for a score of 10 kg, ± 2 kg for a score of 20 kg). Thus, acceptability of the 95% LOA

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 17

depends on the minimal necessary precision for the test score. When using these task

simulations for grading purposes, the 95% LOAs should be taken into account as

passing scores are developed.

Reliability of the tasks were comparable to those observed during reliability of

D
previously evaluated Soldiering task simulations. The learning effect of the sandbag

carry and casualty evacuation are similar to those previously observed during repeated

TE
box and carry (10, 16) and one-repetition maximal box lifts (16). The ICC of 0.79 and

SEM of 5.47% for the 6.4km foot march were similar to the ICC of 0.81 and SEM of 5%

observed during a 3.2-km load carriage trial (16). Likewise, the reliability of the 15-m

casualty drag measured in speed in the present study (ICC 0.90, SEM 11%) were
EP
similar to those observed while dragging a casualty 50 meters (ICC 0.86, SEM 9%)

using time to completion (16). The greater reliability observed during a lift task (casualty

evacuation) than a carry task (sandbag carry) is consistent with previous findings (9).
C

Strengths and Limitations

C

The utility of these task simulations is they provide a method for measuring

Soldier performance in a controlled setting to determine determining the physical

A

capabilities of an individual for graduation assessments. In the field under differing

weather, or on different courses simulating battlefield conditions, the reliability may be

less. Any prior training of Soldiers, soreness or discomfort (both prior to testing or as a

result of the testing), or changes in motivation, especially on the final day of testing,

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 18

could also have an effect. While every attempt was made to control for these factors, it

is unlikely that their influence was completely removed.

Because of recent decision to allow women into the Combat Arms, both males

and females were included in this analysis. While it would have been ideal to run a sex

D
comparison on the reliability of these simulations, we were unable to do so. With the

number of Soldiers, especially women, available for testing, we were underpowered to

TE
run any sex comparisons.

PRACTICAL APPLICATIONS

The simulations used in this study show good reliability and may prove useful
EP
may prove useful for training and assessment purposes. Practitioners should consider

these task simulations when evaluating the physical performance of Soldiers, such as

determining the effectiveness physical training programs. Furthermore, several of these

C

tasks may also be useful in assessing the physical abilities of not only Soldiers, but also

for individuals working in other tactical professions, such as first responders. For best
C

results, participants unfamiliar with the simulations should be provided sufficient

practice.
A

ACKNOWLEDGEMENTS

The authors would like to thank Deborah Gebhardt, Ph.D. and Todd Baker, Ph.D.

for their assistance with the study design. The authors also thank all the researchers

and Soldiers who participated in the data collection.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 19

This research was supported in part by appointments to the Postgraduate

Research Participation Program at the U.S. Army Research Institute of Environmental

Medicine administered by the Oak Ridge Institute for Science and Education.

D
The opinions or assertions contained herein are the private views of the author(s)

and are not to be construed as official or as reflecting the views of the Army or the

TE
Department of Defense. Any citations of commercial organizations and trade names in

this report do not constitute an official Department of the Army endorsement or approval

of the products or services thereof.

EP
The investigators have adhered to the policies for protection of human subjects

as prescribed in Army Regulation 70-25, and the research was conducted in adherence

with the provisions of 45 CFR Part 46.

C
C

REFERENCES

1. Atkinson G and Nevill AM. Statistical methods for assessing measurement error
A

(reliability) in variables relevant to sports medicine. Sports Med 26: 217-238,

1998.
2. Bland JM and Altman DG. Statistical methods for assessing agreement between
two methods of clinical measurement. Lancet 1: 307-310., 1986.
3. Deakin JM, Pelot R, Smith JT, Weber CL, Fortier LD, Rice BL, Fortier CJ, and
Kuhnke TJN. Development and Validation of Canadian Forces Minimum Physical
Fitness Standard (MPFS 2000). Kingston, Ontario: Queen’s University, 2000.
4. Foulis SA, Redmond JE, Warr BJ, Zambraski EJ, Frykman PN, Gebhardt DL,
Baker TA, and Sharp MA. Development of a Physical Employment Testing
Battery for 12B Combat Engineers (Report #16-5). Natick, MA: US Army
Research Institute of Environmental Medicine, 2015.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 20

5. Foulis SA, Redmond JE, Warr BJ, Zambraski EJ, Frykman PN, and Sharp MA.
Development of the Occupational Physical Assessment Test for Combat Arms
Soldiers (Report #16-2). Natick, MA: US Army Research Institute of
Environmental Medicine, 2015.
6. Gebhardt DL and Baker TA. Chapter 13: Physical Performance Tests, in:
Handbook of Employee Selection. JL Farr, NT Tippins, eds. New York, NY:
Routledge, 2010, pp 277-298.
7. Hopkins WG. Measures of reliability in sports medicine and science. Sports Med
30: 1-15, 2000.
8. Knapik JJ and East WB. History of United States Army physical fitness and

D
physical readiness training. US Army Med Dep J: 5-19, 2014.
9. Myers DC, Gebhardt DL, Crump CE, and Fleishman EA. Validation of the Military
Entrance Physical Strength Capacity Test. (Report # 610). Bethesda, MD:
Advanced Research Resources Organization, 1984.

TE
10. Pandorf CE, Nindl BC, Montain SJ, Castellani JW, Frykman PN, Leone CD, and
Harman EA. Reliability assessment of two militarily relevant occupational
physical performance tests. Can J Appl Physiol 28: 27-37, 2003.
11. Payne W and Harvey J. A framework for the design and development of physical
employment tests and standards. Ergonomics 53: 858-871, 2010.
12. Payne WR, Harvey JT, Knez WL, Brotherhood JR, J.F. C, Selig SE, and Ham
DJ. Defence Physical Employment Standards Project. Report 8. Selection of
EP
Criterion Trade Tasks: Infantry and ADG. WR Payne, ed. Ballarat, Victoria,
Australia: School of Human Movement and Sport Sciences, University of
Ballarat, 2005.
13. Rayson M, Holliman D, and Belyavin A. Development of physical selection
procedures for the British Army. Phase 2: relationship between physical
performance tests and criterion tasks. Ergonomics 43: 73-105., 2000.
14. Rayson MP, Bell DG, Holliman DE, Llewellyn M, Nevola VR, and Bell RL.
C

Physical selection standards for the British Army: Phases 1 and 2 (Report
94R036). Farnborough, Hampshire, United Kingdom: Army Personnel Research
Establishment, 1995.
C

15. Sharp MA, Patton JF, and Vogel JA. A database of physically demanding tasks
performed by U.S. Army soldiers (Report No. T98-12). Natick, MA: U.S. Army
Research Institute of Environmental Medicine, 1998.
16. Spiering BA, Walker LA, Hendrickson NR, Simpson K, Harman EA, Allison SC,
A

and Sharp MA. Reliability of military-relevant tests designed to assess soldier

readiness for occupational and combat-related duties. Mil Med 177: 663-668,
2012.
17. Stevenson JM, Bryant JT, Andrew GM, Smith JT, French SL, Thomson JM, and
Deakin JM. Development of physical fitness standards for Canadian Armed
Forces younger personnel. Can J Sport Sci 17: 214-221., 1992.
18. US Army. FM 7-22 Army Physical Readiness Training. Washington D.C.:
Government Printing Office, 2012.
19. Van Ness PH, Towle VR, and Juthani-Mehta M. Testing measurement reliability
in older populations: methods for informed discrimination in instrument selection
and application. J Aging Health 20: 183-197, 2008.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 21

FIGURE LEGEND

Figure 1. Images of A) the simulation deck for the Casualty Evacuation from a Vehicle
Turret; B) the simulator deck for Stow Ammo on an Abrams tank; and C) the simulator
for Load Main Gun on an Abrams tank.

D
TE
EP
C
C
A

Copyright ª 2017 National Strength and Conditioning Association

Table 1. Soldier Characteristics by Cohort
Cohort 1 Cohort 2 Cohort 3
25M/25F 25M/25F 29M/20F
Age years 24.8 ± 4.4 23.4 ± 3.6 23.9 ± 4.0
Height cm 173 ± 10 172 ± 10 174 ± 7
Mass kg 70.4 ± 2.6 68.2 ± 12.5 68.9 ± 11.8
Army Physical Fitness 266 ± 27 270 ± 23 269 ± 21

D
Test Score points
Push-ups #∙2-min-1 55 ± 17 57 ± 15 60 ± 18
Sit-ups #∙2-min-1 69 ± 11 71 ± 10 70 ± 12

TE
Two-Mile Run Time min 15.3 ± 2.2 15.3 ± 1.7 14.9 ± 1.9
Time in Military years 3.1 ± 3.3 3.2 ± 2.9 3.0 ± 2.1
Previously Deployed n 17 20 25
In Combat Arms MOS n 14 9 8
EP
Mean ± SD, M: Males; F: Females
C
C
A

Copyright ª 2017 National Strength and Conditioning Association

Table 2. Performance (Marginal Means ± Standard Error) during Repeated
Measurements of Task Simulations
Task Trial 1 Trial 2 Trial 3 Trial 4
Sandbag
Mean ± SE 2.22 ± 0.11* 2.11 ± 0.10 2.04 ± 0.08 2.06 ± 0.07
Carry min
n 50 50 50 50
Casualty
Mean ± SE 1.10 ± 0.06 1.09 ± 0.06 1.06 ± 0.05 1.05 ± 0.05
Drag m·s-1
n 50 50 50 50

D
Casualty
Mean ± SE 65.8 ± 3.3* 69.8 ± 3.2 70.8 ± 3.2 71.6 ± 3.2
Evacuation
kg n 50 49 49 49
Move Under

TE
Mean ± SE 2.29 ± 0.04* 2.26 ± 0.03 2.24 ± 0.04 2.25 ± 0.04
Fire min
n 49 49 46 43
Stow Ammo
Mean ± SE 3.55 ± 0.42* 4.47 ± 0.44* 4.87 ± 0.42* 5.36 ± 0.41
rounds·min-1
n 49 49 46 43
Load Main
Mean ± SE 26.1 ± 1.2* 21.0 ± 1.0* 18.9 ± 0.8* 17.4 ± 0.7
EP
Gun s
n 50 49 46 44
FAASV
Mean ± SE 2.45 ± 0.22* 3.15 ± 0.27* 3.47 ± 0.27 3.64 ± 0.27
rounds·min-1
n 47 46 45 41
Foot March
Mean ± SE 81.1 ± 1.7 80.0 ± 2.4 76.5 ± 2.1 78.7 ± 2.0
min
C

n 49 48 44 33
* Indicates significantly different from consecutive trial p < 0.05
C
A

Copyright ª 2017 National Strength and Conditioning Association

Table 3. Relative and Absolute Reliability of Task Simulations
Relative Absolute
Trial 95%
Task n ICC (2,1) SEM 95%
Comparison Ratio
[95%CI] (%) LOA
LOA
Sandbag Carry 0.87 0.27
50 1 vs 2 0.75
min [0.78-0.92] (12%)
0.85 0.25
2 vs 3 33%
[0.75-0.91] (12%)

D
Casualty Drag 0.90 0.13
-1 50 1 vs 2 0.35
m·s [0.83-0.94] (11%)
Casualty 0.94 6.9
49 1 vs 2 15.0

TE
Evacuation kg [0.90-0.97] (10%)
0.96 4.2
2 vs 3 11.7
[0.94-0.98] (6%)
Move Under 0.90 0.08
49 1 vs 2 0.21
Fire min [0.82-0.94] (3%)
0.93 0.06
46 2 vs 3 0.16
[0.88-0.96] (3%)
EP
Stow Ammo 0.96 0.58
49 1 vs 2 1.60
rounds·min-1 [0.93-0.98] (15%)
0.97 0.51
46 2 vs 3 1.42
[0.94-0.98] (11%)
0.96 0.57
43 3 vs 4 1.59
[0.92-0.98] (11%)
Load Main Gun 0.84 2.82
C

49 1 vs 2 33%
s [0.73-0.90] (12%)
0.90 1.92
46 2 vs 3 27%
[0.83-0.94] (10%)
C

0.93 1.36
44 3 vs 4 21%
[0.88-0.96] (7%)
FAASV 0.93 0.40
46 1 vs 2 40%
Rounds·min-1 [0.88-0.96] (15%)
A

0.93 0.44
44 2 vs 3 1.22
[0.88-0.96] (14%)
0.93 0.45
42 3 vs 4 1.25
[0.88-0.96] (13%)
Foot March 0.76 5.89
48 1 vs 2 16.34
min [0.61-0.86] (7%)
Shaded trials indicate learning effect present. Unshaded trials indicate reliability in
absence of learning effect.

Copyright ª 2017 National Strength and Conditioning Association

 Reliability of Simulations 1

Figure 1
A)

D
B)

TE
EP
C)
C
C
A

Copyright ª 2017 National Strength and Conditioning Association