See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/265022847

The ARTEMIS Air-to-Air Combat Model

Article

CITATION READS

1 106

7 authors, including:

Clinton Heinze
Australian High Commission, London, United Kingdom
36 PUBLICATIONS 453 CITATIONS

SEE PROFILE

All content following this page was uploaded by Clinton Heinze on 06 January 2015.

The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document
and are linked to publications on ResearchGate, letting you access and read them immediately.
 The ARTEMIS Air-to-Air Combat Model
Clinton Heinze; Brian Hanlon1; Michael Turner; Kelvin Bramley;
John Rigopoulos; David Marlow; Kurt Bieri2
Air Operations Research Branch
Air Operations Division
Defence Science and Technology Organisation
firstname.lastname@dsto.defence.gov.au

Abstract. Operations Research is an important element supporting the decision process in major Air Force
acquisition programs and must often address itself to a range of issues of varying complexity. An issue of
particular complexity is air-to-air combat. Air-to-Air combat is a highly dynamic interaction between, possibly
several, high performance aircraft employing sophisticated weapon and sensor systems and applying novel
tactics designed to yield an advantage against the opponent. The combination of numerous entities, multiple
system capabilities and complex tactics makes the air-to-air combat problem an especially challenging area of
military operations research. Currently, Australia is evaluating the capabilities of the F-35 Joint Strike Fighter
(JSF) towards possible acquisition of this aircraft to replace Australia's ageing F/A-18 and F-111 fleets.
Evaluation of the air-to-air combat performance of the F-35 JSF is a central element of this assessment.
Building on the body of analytical studies completed in previous study phases, this evaluation will employ a
new air-to-air combat simulation model currently under development, the AiR Tactical Engagement MIssion
Simulator (ARTEMIS) to explore issues specific to the F-35 JSF. The requirement for such a model follows an
assessment of available air combat models in Australia and overseas. ARTEMIS is based on a flexible
simulation architecture and employs teamed intelligent agent technology to model complex multi-aircraft
tactics and Command & Control structures. This paper will describe the ARTEMIS model and the context of
its development. The paper will then be extended to a consideration on how ARTEMIS will be employed in
operations research and the issues that will be addressed.

partly a project management story; and points to a

1. THE HUNT CONTINUES
ARTEMIS is the latest in a series of simulators
developed by Air Operations Research Branch
(AOR) of the Defence Science and Technology
Organisation (DSTO). Rather than report only on
the technology behind ARTEMIS (Section 2), this
paper will also provide: insights into the
development of a simulation where there is a large
and diverse stakeholder base (Section 3) including
a discussion of some of the project management
challenges; the purpose to which the simulator will
be put and a sample scenario (Section 4); an
indication of future directions for ARTEMIS and
related technology (Section 5).
1.1 Four and Counting
In the lineage outlined below ARTEMIS (and
other simulation systems currently under
development at AOR are representative of fourth
generation systems. Over the course of a decade
the capability of AOD to develop and deploy large
simulations has improved. The maturation of this
capability is a software engineering story, at least
change in the ways that AOR does business that
have improved verification and validation and
taken AOR in a direction toward experimentation
and the study of future warfighting concepts (such
as network centric warfare). This paper sets out to
tell this story in the context of ARTEMIS, the
latest and (arguably) most sophisticated simulation
system as yet developed by AOR.
ARTEMIS’ Parentage
PACAUS (1988-1998): sophisticated air combat
model that proved highly effective for modelling
small numbers of aircraft in BVR and WVR
combat. Coded almost entirely in FORTRAN with
very little thought given to architectural design, as
it grew it became increasingly more difficult to
modify and maintenance was all but impossible.
SWARMM (1995-2002): saw the introduction of
intelligent agents and the commencement of an
effort to improve software engineering, V&V and
build a closer relationship with subject matter
experts. Gains included the flexible representation
of tactics; quicker development times; and a

1
Brian Hanlon has since left DSTO and now works for the Department of the Treasury of the Australian Government.
2
Kurt Bieri is a KESEM International employee engaged by DSTO as the Software Project Manager for the ARTEMIS project.
significantly more modular and simpler to
maintain simulation.
AEW&C BattleModel (1998- ): the requirements
of the Project Wedgetail tender evaluation process3
resulted in AOD developing BattleModel. A
considerable improvement in robustness resulted
from the improvement in the software engineering
practices that surrounded the project. The gradual
maturing of software engineering was pushed by
the tight schedule and hard deadlines of a major
Air Force acquisition project that allowed zero
tolerance for development delays. Unsurprisingly
there were a number of side benefits.
1.2 Modelling for Operations Research
Simulations developed by AOR are for operations
research: advice to the acquisition process;
development of operational concepts and tactical
procedures; and the general gaining of knowledge
about current and likely future air operations.
These simulations often facilitate the exploration
of large parameter spaces and assist in developing
a statistical understanding of the nature of the
operations being studied.
Performance
Typically many thousands of simulated missions
are flown in any given study and, as the simulated
timescales can be several hours, performance often
becomes a significant issue. Within the typically
available time and computational resource limits,
simulation performance of much faster than real
time is often a requirement.
Level of Representation
In any modelling and simulation endeavour there
are decisions made about the level of
representation that is required. For operations
research simulation, and consequently ARTEMIS,
there is a trade-off to be made between level of
detail, computational performance and cost. It is
important to note here that any given scenario
might involve as many as sixteen aircraft.
Furthermore, the scenarios of interest to
ARTEMIS do not necessarily require the highest
levels of representation in the models.
Sensitivity Analysis
Sensitivity analyses can reveal those areas of the
simulation that will have the greatest impact on the
outcome. A simple example will illustrate this
point. A study might be considering the
effectiveness of a new short-range air-to-air
missile. This missile will be launched well inside
the normal detection range of the radar and often
after extensive combat manoeuvring and whilst
3
Project Wedgetail was the acquisition and introduction into
service of an Airborne Early Warning and Control
Aircraft.
within visual range. The effectiveness of the
missile under these conditions will be sensitive to
the missile performance, the target and launch
aircraft manoeuvring and any countermeasures. A
highly sophisticated radar model will add nothing
to results and will impact on the computational
performance. It is necessary to select the level of
representation appropriate for the studies being
undertaken, sensitivity checks can ensure this and
give indications of error bounds over any reported
measures of effectiveness.
Requirements Volatility
Within these requirements that drive simulations
like ARTEMIS there are always compromises that
must be made due to unavailability of data,
uncertainty in requirements, or the use of legacy
systems. The representation of an aircraft whose
systems are still under development requires
flexibility in the representation so that as the
design firms up the models can be tailored to
match. This is also true of the models of the
tactical employment of those systems.
Third Party Components
The core-business of AOR is to provide advice
across a broad spectrum of matters related to air
operations. Maintaining the level of technical
knowledge required for all of the modelling and
simulation activities is beyond the capabilities of
the relatively small team. Other branches of
Defence assist with the provision of technical
advice, data, and operational knowledge about
specific hardware. Provided knowledge might be
limited to background technical briefings or as
thorough as completely documented software
models. In this way AOR outsources some of the
simulation development activities to appropriately
skilled and resourced third parties. Validation of
the model can then be vested with those third
parties. A consequence of this approach is that
AOR requires an approach to simulation
development that is flexible enough to
accommodate third party software components as
they become available.
1.3 Risk
ARTEMIS combines a suite of technologies and
architectural design choices that have resulted in
the most sophisticated air combat OR simulation
yet deployed by AOR. Despite AOR’s capability
maturity, the innovative nature of some of the
design choices has resulted in a system that is as
technologically sophisticated as any deployed by
AOR but has the added pressures of a large and
diverse development team, tight budgets and
schedules and a base set of requirements that
continue to display some uncertainty. The
following section outlines the technologies in use
in ARTEMIS and very brief descriptions of the
relevant architecture.
2. TECHNOLOGY
This section outlines the core technologies. These
technologies are related to the kernel or simulation
architecture; the tactical reasoning component; the
physical systems modules; and the interfaces.
2.1 Simulation Architecture
The simulation architecture in use in ARTEMIS is
BattleModel. BattleModel was designed to provide
a capability to incorporate third-party software
components into a larger simulation. Models
sourced from elsewhere within DSTO or from
other organisations can be obtained and bundled
together with the existing library of models to
create a complete simulation.
Faster than real time performance
Unlike human in the loop training simulators OR
requires faster than real time performance – often
much faster than real time performance in order
that they produce the many thousands of runs
necessary for a detailed statistical analysis.
Repeatability
The simulations that are described here generate
knowledge: understanding of tactics; knowledge of
the manner in which systems interact; and
measures that indicate the relative performance of
various systems. As such they provide answers to
questions and, fundamentally, those answers must
be explainable and justified. Inevitably this leads
to a stronger sense of validation than might be
required for training simulators where the
behaviour need only be perceived as correct to be
beneficial. OR simulations must be explainable,
visible, repeatable and trusted in ways that other
systems might not need to be.
Modularity
Modularity, a generally desirable property of
software, influences OR simulations quite
strongly. Making choices in the performance/level
of representation/cost trade-off will commonly
result in requirements for models of varying
complexity. A simulation architecture is required
that supports the substitution of models at different
points on the performance/level of representation
trade-off spectrum.
2.2 Tactical Reasoning
The tactical reasoning in ARTEMIS makes use of
JACK intelligent agent technology.
JACK
JACK4 is an intelligent agent programming
language, based on the BDI agent model of Rao
and Georgeff [4]. In many respects it is successor
to PRS and dMARS, but is implemented in JAVA
providing greater portability.
Observe Orient Decide Act (OODA)
ARTEMIS uses a design pattern that has proved
useful in several simulations [2]. During the
development of SWARMM it proved useful to
partition the tactical reasoning in a way that
simplified the conceptualisation its functioning in
keeping with nature of the domain. Boyd’s
OODA loop model of military decision making
was preferred for several reasons: it was familiar
to Air Force fighter pilots and matched closely
with their introspective account of tactical decision
making; it can be implemented within languages
like JACK in a manner that is consistent with
time-stepped simulation; and it provides a
simplifying layer on top of the complex semantics
of the BDI model in a manner that can enforce
repeatability.
The particular implementation of the OODA loop
model serialises the tactical decision making on a
time step by time step basis into four components.
The first “O” is a currently sparsely populated
component responsible for handling situation
awareness. This is really little more than a
mapping from the data made available by the rest
of the simulation environment into the form
required by the agent. The second “O” (orient)
includes most of the processing of the current state
to provide situation and threat assessment.
“Decide” deals mainly with the agents reasoning
about a course of action to select and finally the
“Action” provides the implementation of the
standard operating procedures and tactical plans.
TEAMS
A significant challenge in modelling air combat is
simulating the complex team behaviours
associated with pairs and fourship tactics. JACK
provides some basic functionality for modelling
teams and recent additions have added
sophistication. This has changed the manner in
which team tactic modelling is undertaken.
Previously, hand coded infrastructure was
necessary for modelling teams [3]. Tidhar [5]
defined the semantics of a model that provides
representation of team tactics and views the team
as an explicit and separate entity.
In ARTEMIS a pair is represented by three agents,
one for each of the individual aircraft and one that
represents the pair. This pair entity encapsulates
any aspects of the reasoning that are associated
4
JACK is a product of Agent Oriented Software, further
information is available from www.agent-
software.com
with coordinated activity. This is often range of information are available throught the
conceptually difficult but offers knowledge and GUI.
software engineering advantages. A variant of the
OODA loop that caters for team decision-making
has been developed specifically for ARTEMIS.

2.3 Physics Models

The latest fighter aircraft – the JSF being a
prototypical example – are fitted with a number of
new systems or technologies. The radar models
are an interesting case in point. The JSF and most
latest generation fighter aircraft are fitted with
electronically scanned radars. This represented a
new challenge for AOR branch. All prior
modelling of electronically scanned radars had
been confined to early warning and surveillance
aircraft. A range of similar technical modelling Figure 2: XCombat is a three dimensional
challenges was met for the electronic support visualisation tool in wide use in AOR. The
measures, the aerodynamics, radios and data-links, package provides a relatively light-weight and
weapons and countermeasures. standard GUI for visualising air combat in three
dimensions. Out-of the cockpit and god’s eye
modes are available. It is likely to be incorporated
2.4 Interfaces
into ARTEMIS in a future release.
ARTEMIS provides the user with a number of
Tactical Reasoning
important interfaces. One of these is a traditional
graphical user interfaces for visualising air The standard JACK GUIs are part of ARTEMIS.
combat, one allows the visualisation of the tactical They are mentioned here because AOR branch has
reasoning component during the execution of the invested in the development and improvement of
scenario and the last, but most important, is a data the guis with features that have been custom-built
collection module that allows the analyst to gather (see screenshots). The ability to program the
relevant data from the simulation. tactical reasoning in a graphical manner has in the
past provided advantages for intelligent agent
MOE Module
development and there is long term expected
Unlike many simulators, BattleModel’s primary benefits for ARTEMIS: validation and
interface is not graphical. The measures-of- verification; and building trust with subject matter
effectiveness (MOE) module is programmed to experts and operational military personnel.
provide basic data gathering and first cut analysis
from a batch set of runs.
Scenario Initialisation and Visualisation
The scenario initialisation and visualisation guis
available as a bundled part of BattleModel are
included as a part of ARTEMIS. (see screenshots)

Figure 3: A screen shot of the JACK

Development Environment. By utilising a
programming language that expresses the tactics in
a graphical form with natural language
descriptions it is possible to ease the validation
process and improve the understanding of the
Figure 1: BattleModel provides the BattleVision
code.
2D interface for visualising scenarios. Control of
the simulation, playback and record, and a wide
3. PROJECT MANAGEMENT
The ARTEMIS project is sponsored through the
ADO by the NACC project office who are the
ultimate customer for the results of any studies.
Responsibility for the development of the
simulation lies with AOR as does the ultimate
sign-off on the validity of the any results generated
and hence the validity of the simulation.
AOR have contracted project management support
for the project and, furthermore, employ
contractors for the development and support of
simulation architecture (KESEM) and for the
development of large components of the tactical
reasoning (Agent Oriented Software).
3.1 Stakeholders, responsibilities and
obligations
AOR has approximately 9 Commonwealth groups
providing input into ARTEMIS development and
two contractor organisations (AOS and KESEM)
which are undertaking the development. In
addition, as previously, mentioned ARTEMIS is
also faced with challenging schedule and cost
constraints. These diverse project interfaces and
constraints require amore formal engineering
processes than have previously been applied. The
processes include such things as the formalisation
of all model requirements, configuration
management, quality assurance and appropriate
levels of software engineering rigour; and combine
to provide confidence that the project constraints
can be met.
3.2 V&V
System level validation and verification is
ultimately the responsibility of AOR. Some
models, particularly those models that represent
the performance of aircraft sub-systems are
validated at the component level by DSTO
technical divisions with expertise in relevant
technologies.
The validation of the entire simulation system is
undertaken by AOR in a variety of ways:
system-level regression testing against
previously developed and validated air
combat models;
model-level validation against empirical
data obtained from a variety of sources;
sanity checks against the expertise of
AOR branch;
future plans for experimentation and
consequent validation by Air Force
personnel in the style of AOR’s previous
validation activities [1].
4. STUDIES
With the understanding that important but
classified study details must remain unreported in
this open forum the following are indicative of
ARTEMIS questions and scenarios.
The New Air Combat Capability (NACC) project
In June 2002, Australia entered the System
Development and Demonstration (SDD) phase of
the JSF project with the proposed intent of
acquiring the aircraft as a replacement of the
capability currently provided by the F/A-18 and F-
111. As a result, the New Air Combat Capability
(NACC) project was created to determine how the
JSF would be integrated into the Australian
Defence Force (ADF). Consequently AOR has
been tasked to address the following key
questions:
Can the JSF meet the Australian
requirements for air combat and strike
capability?
How many JSF aircraft are required by
the ADF?
What and how many adjunct systems are
required to complement the JSF (Tankers,
AEW&C etc.)?
The initial application of ARTEMIS will be to
evaluate the effectiveness of the JSF in air combat
missions, which will ultimately contribute to
addressing the key questions of the NACC project.
Such evaluation studies will explore variations on
a number of parameters to determine the
circumstances under which operational
effectiveness hold. These parameters are primarily:
Air combat missions, such as Defensive
Counter Air (DCA) and Offensive
Counter Air (OCA)
Team sizes, such as 2-ship and 4-ship
formations
Operational concepts, ranging from
current RAAF tactical procedures to
novel concepts. Novel concepts will take
advantage of the revolutionary systems
technology inherent on the JSF. For
example, stealth design, active and
passive sensors, and data-links enabling
Network-Enabled Concepts with teams of
JSF and with AEW&C as an adjunct
system.
Threat aircraft types projected for the
timeframe in which the JSF would be in
service.
Example Study Question
In air combat, the fighter that is able to detect,
target, and launch a Beyond Visual Range (BVR)
weapon against the opposing fighter earliest has an
advantage in winning the engagement. Therefore,
an important question is:
How do the JSF systems capabilities and the way
they are employed contribute to the ability to
achieve first-look and first kill?
Key drivers to this question are the ability of the
JSF to have Situation Awareness of the threat and
deny Situation Awareness to the threat during the
intercept. The former depends on how the sensor
systems are employed, whether active or passive,
on-board JSF, off-board JSF, or adjunct AEW&C.
The latter depends on the nature of the emitted and
reflected signatures the JSF presents to the threat
sensors.
Sample Scenario
An example study is a 2v2 scenario of offensive
counter air (OCA) versus defensive counter air
(DCA). The JSF and threat fly to a common
waypoint and may employ similar offensive and
defensive tactics. Various cases are investigated
which parameterise the JSF and threat system
performance and tactics. Each case involves
Monte-Carlo runs, each with a randomised initial
threat flight paths. The statistical engagement
outcomes for each case are then evaluated and
analysed.
Reporting and Advice
The result of these studies will contribute to a
report providing Science and Technology (S&T)
advice to support Defence and Government
approval of the JSF acquisition.
5. WHEN THE WAR IS OVER
This section documents the current capability of
ARTEMIS, some of the more important lessons
learned in the development to date, and briefly
maps the future development.
5.1 Current Capability
ARTEMIS is part way through its development
process and development is currently ongoing. The
software is at a stage where studies to commence
and useful results obtained. The first round of
studies will be reported by the middle of this year.
Current implementation includes a variety of
simulations of physical systems modelled at the
appropriate level. The models of fighter pilots are
implemented in JACK and these include a
formalising of many of the ad-hoc design
approaches used in earlier air combat models. An
example of this is the four box model, a
computational version of the OODA loop model of
reasoning underpins the decision cycle of the
intelligent agents. ARTEMIS is the first air combat
model developed by AOR to take advantage of the
teaming extensions that have been developed by
View publication stats
AOS and AOR during the course of the last five
years.
5.2 Lessons Learned
There will always be a need to trade-off
computational performance, level of representation
and cost. This is perhaps even more critical in OR
simulations than in real-time training systems. The
requirements of the studies (and indicative
scenarios) should drive the development of the
models. This is particularly true of the tactical
decision making models. If the requirements are
not firmly pinned down there is little choice but to
build for flexibility but there is a price to pay.
Even though this is a fourth generation system and
AOR have had extensive development experience
with similar systems, the new technology (JACK,
BattleModel) and a changed management structure
involving greater contribution from contractors
have combined to create relatively high levels of
risk.
ARTEMIS will be used now and into the future to
meet the operations research simulation studies
requirements of the ADF. Development is
ongoing to meet the tight schedules involved.
ARTEMIS has the flexibility to meet the needs of
future air combat studies in AOR and will
underpin the major new Aerospace Battlelab
Facility.
REFERENCES
[1] Clint Heinze, Martin Cross, Simon Goss, Torgny
Josefsson, Ian Lloyd, Graeme Murray, Michael
Papasimeon, and Michael Turner. Agents of
change: The impact of intelligent agent technology
on the analysis of air operations In L. Jain, N.
Ichalkaranje, and G. Tonfoni, editors, Advances in
Intelligent Systems for Defence, volume 2 Series
on Innovative Intelligence, chapter 6, pages 229---
264. World Scientific, River Edge, New Jersey,
USA, 1 edition, December 2002.
[2] Heinze, C. et al., (2002) “Interchanging Agents
and Humans in Military Simulation”, AI
Magazine, 23:2, pp37-47
[3] G. Tidhar, C. Heinze, and M. Selvestrel. Flying
Together: Modelling Air Mission Teams. Applied
Intelligence, vol. 8, pp. 195-218, 1998.
[4] A.S. Rao and M.P. Georgeff. Modeling rational
agents within a BDI-architecture. In J. Allen, R.
Fikes, and E. Sandewall, editors, Proceedings of
the Second International Conference on Principles
of Knowledge Representation and Reasoning
(KR'91), pages 473--484. Morgan Kaufmann,
1991.
[5] Tidhar, G. Team Oriented Programming: Theory
and Practice. PhD Thesis, University of
Melbourne, 1999.