Intelligent Agent Technology

Jeffrey M. Bradshaw
Bob Carpenter
Rob Cranfill
Mark Greaves
Heather Holmback
Renia Jeffers
Luis Poblete
Amy Sun

Applied Research and Technology

Shared Services Group
The Boeing Company
jeffrey.m.bradshaw@boeing.com
 Why Software Agents?
◆ Original agent work instigated by researchers studying distributed
intelligence

◆ New wave of agent research motivated by two practical concerns:

– Overcoming the limitations of current user interface approaches
– Simplifying the complexities of distributed computing

◆ Though each of these problems can be solved in other ways, the

aggregate advantage of agent technology is that it can address both of
them at once:
– by supplementing direct manipulation with indirect management
approaches
– by building in high-level, loosely-coupled collaborative
capabilities “out of the box”
Evolution of System Connectivity

Disjoint

Ad hoc

Encapsulated
Cooperating Systems with Single
Agent as Global Planner

A
Cooperating System with
Distributed Agents
A

A A

A A
 Agent-Enabled System
Architecture
Integrated 
interface to 
knowledge 
media

Agents as  Agent as 
intelligent  personal 
interface  assistant
managers

Agent­to­agent 
communication Agents 
behind the 
scenes

Interapplication communication
 What is a Software Agent?
◆ Agents are software entities that function continuously and
autonomously in a particular environment that is often inhabited by
other agents and processes

◆ Ideally a software agent should be able to:

– carry out activities without requiring constant human guidance
– learn from its experience
– communicate and collaborate with people and other agents
– move from place to place over a network as necessary

◆ Not all software agents need be “intelligent” (agents vs. minions)

◆ There is no hard dividing line between object technology and multi-

agent technology
 Basic Agent Characteristics
Agents act autonomously to
accomplish objectives.
• Goal-Directed
• Knowledgeable
• Persistent
• Proactive & Reactive Autonomous
Agents cooperate to
achieve common goals.
• Communication Protocols
• Knowledge-Sharing
Agents adapt to their • Coordination Strategies
environment. • Negotiation Protocols
• Dynamic Interaction
• Alternate Methods Adaptive Cooperative
• Machine Learning

Note: Agents can be either static or mobile, depending on

bandwidth requirements, data vs. program size,
communication latency, and network stability
(Dyer, DARPA CoABS)
 Agents and Objects
Objects Agents
Basic unit instance agent

State­defining parameters unconstrained knowledge, desires,

intentions, capabilities,…

Process of computation operations messages

Message types defined in classes defined in suites

Message sequences implicit defined in conversations

Social conventions none honesty, consistency,…

(Adapted from Shoham)
 Applications of Software Agents
◆ Office automation/engineering support
– mail filtering
– meeting scheduling
– intelligent assistance
– training and performance support
◆ Information access
– retrieval, filtering, and integration from multiple sources
– Internet, intranet, extranet
◆ Resource brokering
– “fair” allocation of limited computing resources
– dynamic rerouting and reassignment of tasks
◆ Active document interfaces
– intelligent integration and presentation to suit the task
– dynamic configuration according to resource availability and platform constraints
◆ Intelligent collaboration
– between systems
– among people
– mixture of people and agent-assisted systems
 Boeing IAT Program Objectives
u More powerful agent frameworks
– New KAoS release
– UtterKAoS: Conversations, Security, Persistence, Mobility,
Middle Agents, Planning
– Incorporation of COTS components (e.g., Voyager, Java platform
enhancements)
u Easier creation of sophisticated agents
– ADT, comprised initially of CDT, SDT, PDT
u Deploy in spectrum of application areas
– Current areas: Information Access, DIG-IT, NASA Aviation
Extranet, DARPA JumpStart
– New opportunities: Spacecraft autonomy, hybrid networking QoS,
security, UCAV, engineering, manufacturing
 Some Long-Term Requirements for
Industrial-Strength Agents
◆ Architecture appropriate for a wide variety of domains and
operating environments
◆ Hardware-, operating-system-, programming-language-
independent
◆ Separability of message and transport layers
◆ Foundation of distributed-object/middleware
(e.g.,CORBA, DCOM) and Internet technologies
◆ Fits well into component integration architectures (e.g.,
ActiveX, JavaBeans, Web browsers)
◆ Principled extensibility of agent-to-agent protocol
◆ Designed to work with other agent architectures, and to
allow easy “agentification” of existing software
◆ Must be able to incorporate agent interoperability
standards as they evolve
 KAoS Implementation Context
Adaptive Virtual 
Document

Database 
Component integration framework T Component
SGML/XML 
Component Multimedia 
Component

Agents

Object Request 
CORBA
Broker 

Web and  Link  Component 

other  Servers tools and 
Local and remote databases and services Internet  Fine­grained  services
services data objects
KAoS Structure and Dynamics
Birth

Life Agent Structure
 
Update  • Knowledge
Structure    • Facts
   • Beliefs
• Desires
Formulate/Act on  
• Intentions
Intentions
• Capabilities

Death
Cryogenic  
State
KAoS Extension and Generic Agent

Agent Extension

Specific to  Optional  Various 

Particular  Planner Capabilities
Agents

Shared by  Conversation  Transport­Level 

All Agents Support Communication
Security
Generic Agent
Agent-to-Agent Communication Within an
Agent Domain
Agent Domain

Generic
Agent
Instance Agent B
Agent A
Agent-to-
Agent Generic
Protocol Agent
Instance
 Domain Manager and Matchmaker
The Domain Manager:
◆ Controls entry/exit of agents within a domain, governs proxy agents (i.e., security)
◆ Maintains a set of properties on behalf of the domain administrator
◆ Provides the address of the Matchmaker to agents within its domain (i.e., naming)
The Matchmaker:
◆ Helps clients find information about the location of agents that have advertised
their services
◆ Forwards requests to Matchmakers in other domains as appropriate
◆ Can be built on top of native distributed object system services (e.g., trader)
Agents Providing Services:
◆ Advertise their services to the Matchmaker
◆ Are notified by the Matchmaker if their services have been registered
◆ Withdraw their services when they no longer wish to provide them
Agents Requesting Services
◆ Ask the Matchmaker to recommend agents that match certain criteria
◆ Are given unique identifiers for the agents that match the criteria
◆ Communicate directly with these agents for services
 Anatomy of a KAoS Domain
External
Telesthetic Resource
Extension Proxy to
Another
KAoS
Mediation
Domain
Extension
GA
GA
Proxy
Extension
KAoS Agent
Domain GA
Mgr. Extension Domain
GA Ext. from
Foreign
Matchmaker Domain
Extension
GA Adapter

GA

GA = Generic Agent
 Conversations
◆ Social interaction is more appropriately modeled when conversations
rather than isolated illocutionary acts are taken as the fundamental unit
of discourse
◆ Two approaches to implementing agent conversations (Walker and
Wooldridge):
– off-line design: social laws are hard-wired in advance
– emergence: conventions develop from within a group of agents
◆ KAoS currently provides only for off-line design of conversations,
represented as state-transition networks
– Shared knowledge about message sequencing conventions enables agents
to coordinate frequently recurring interactions of a routine nature simply
and predictably.
– Cohen and Smith’s semantics and joint intention theory have been used to
analyze KAoS conversation policies
– In the future, more sophisticated agents will either be able to use less
constraining “landmark-based” conversation policies or fall back to more
rigid policies with identical semantics to communicate with simpler
agents
– In support of this, DARPA is funding us to develop a Conversation-
 KAoS Conversation Policies
u Interaction among agents best modeled at the conversational
level, rather than isolated speech acts
u Conversation policies are agent dialogue building-blocks that
provide a set of constraints that define and restrict what can take
place in individual agent conversations
– Policies can be expressed via many different representation formalisms,
from regular expression grammars to dynamic logics
u Conversation policies ensure reliable communication among
heterogeneous agents while lessening agent’s burden of
inference
– Agents choose between a greatly reduced number of possible
conversational moves
– Conversation manager (component of “generic agent”) assures
compliance with policy; handles exceptions
u References: http://www.coginst.uwf.edu/~jbradsha/
 “Conversation for Action” Policy
B->A: Report
A->B: Request B->A:Promise Satisfied
1 2 A->B: Decline 3
Report A->B: Accept
B->A:
B->A: B->A: Report
Counter
Decline Renege
A->B:
A->B:
9
Counter
Accept 6 4
A->B: 5 A->B:
Withdraw
A->B:
Withdraw
Withdraw
A->B:
Withdraw
7 8
• Communication about commitments (promise, renege) is handled explicitly, and
A can notify B when the request was not fulfilled to its satisfaction (decline report)
• See formal analysis of Conversation for Action Policy in Smith and Cohen 1996
AAAI paper
 KAoS Applications
◆ DIG-IT: Boeing digital data integration effort to integrate
agents in next-generation PMA and BOLD
◆ NASA Aviation Extranet: Agent-assisted access to
information and services over a large-scale virtual private
network
◆ AHCPR CDSS Project: Long-term follow-up support for
bone marrow transplant patients at the Fred Hutchinson
Cancer Research Center
◆ DARPA Jumpstart Project: Development of agent design
toolkit (Boeing, UWF Cognition Institute, Sun
Microsystems, IntelliTek)
◆ Agents for space applications: Proposal to use KAoS for a
multi-agent testbed in satellite operations, and in the
development of a Personal Satellite Assistant (in
preparation)
 JumpStart Project Overview
u Selected under the DARPA CoABS Program
– Approximately 20 other participants
u Partners: Boeing , Sun, UWF, IntelliTek
u Collaborator: Oregon Graduate Institute (CHCC)
u Deliverables:
– Prototype software (CDT and SDT)
– Periodic technical reports and demos
– Interoperability demos with other CoABS participants
DARPA’s Vision of the Future of
Agents
u The Future of Agent Ensembles
– Agents authored by different vendors at different times
– Wide variety of agent reasoning and action capabilities
– Complex operational environment:
• Unpredictable universe of action
• Dynamic task-specific agent teams
• Collaborative, negotiated problem-solving behavior
u The Future of Agent Developers
– More agents written by domain experts; fewer agents written by
agent-technology experts
– Decreased ability to control agent contexts of use
 Simple Agents May Not Need a
Complex Theory
u Simple agent systems may require only simple models of
communication to achieve their ends
– Limited tasks, collaborations, interactions with one another
– Predictable all simple-agent universe of action
– Limited and domain-specific reasoning requirements
– Conversations are atomic transactions
u Example:
– Simple personal information retrieval agents
• interact mainly with non-agent information sources
• little negotiation or bargaining
 Sophisticated Agents Require
Sophisticated Theory
u But, consider more complex applications, involving:
– Higher reliability, verifiability, precision of expression
– Arbitrary, dynamic agent collaboration with negotiation
– Unpredictable universe of action
– Complex autonomous reasoning about other agents, plans
– Extensive human-agent interaction
u Examples:
– Electronic Commerce/Electronic Trading, Air Traffic Control,
Health Care, Military, etc.
u This requires a sophisticated multiagent communication
model, e.g., conversations, with an explicit semantic
foundation.
 Operating in Heterogeneous
Environments
“What We’ve Got Here is a Failure To Communicate”

◆ Mixture of different agent frameworks

◆ Mixture of simple and sophisticated agents
◆ Approach: shared conversation and security policies,
generated off-line, that increase interoperability and
robustness in heterogeneous agent environments
 JumpStart Technical Objectives
◆ Current Focus: Communication and Security tools
– More agents written by domain experts; fewer agents written by agent-technology
experts
– CP scenarios require that agent policy configuration be rapid and robust
– Additional classes of development tools needed in future
◆ Help developers design reliable agent conversations
– Help develop ACL semantic and pragmatic theory and standards
– Provide a prototype conversation design tool (CDT)
• Aid agent developers in understanding ACL semantics
• Help select, specialize or generate appropriate conversation policies
◆ Help developers design reliable systems with desired agent security
characteristics
– Develop foundations for agent security and mobility standards
– Provide prototype security design tool (SDT) allowing agent developers to easily
select, specialize or generate appropriate agent security policies
 Conversation Policy Example:
Winograd and Flores CFA
B->A: Report
A->B: Request B->A:Promise Satisfied
1 2 A->B: Decline 3
Report A->B: Accept
B->A:
B->A: B->A: Report
Counter
Decline Renege
A->B:
A->B:
9
Counter
Accept 6 4
A->B: 5 A->B:
Withdraw
A->B:
Withdraw
Withdraw
A->B:
Withdraw
7 8
 Combining Finite-State-Based
and Plan-Based Conversation
Policy Approaches
u Intelligent agents can use less constraining plan-based
policies that give them flexibility of determining many
specifics of conversational moves on-the-fly
u Constraints governing plan-based conversation policies
make them less complicated to implement than
unrestricted agent dialogue models
u Simpler agents will continue to rely on more rigidly
defined FSM-based policies where the universe of possible
moves has been pre-computed “off-line”
u FSM and plan-based versions of same policy must comply
to same semantics and pragmatics
u Appropriate “version” can be negotiated between agents at
runtime
Extending Semantics/Pragmatics
u Participate in ongoing ACL development
– KAoS, AgentTalk, FIPA, KQML-Lite, KQML-Rite
– Ultimate goal of consensus on a compositional semantics with
principled extensibility
u Analyze the ACL speech acts & conversation policies
– We will study/develop basic conversation properties (e.g., the
ordering, timing, sequences of communication acts)
– Match representations of conversation policies to diverse levels of
agent capability:
• Finite-state-machine models
• Landmark models
• Emergent conversations
– FSM and landmark models of same policy must comply to same
semantics and pragmatics; choice of model negotiated at runtime
between agents
– We will also investigate other pragmatic conditions imposed by
context (e.g., meta-conditions on agent conversations)
 CDT: An Extensible Java Toolkit
for Agent Conversation Design
u The CDT is a formal design and verification system for a
given theory of agency and ACL
u Stanford’s OpenProof will be the core framework
– OpenProof is a component-based (JavaBeans) formal heterogeneous
reasoning environment
• Allows development of various representations (sentences, reasoning trees,
FSMs, Dooley graphs, Petri nets, etc.)
• Logical fragments (deductive rules, theorem-provers)
• Heterogeneous transfer rules
– Extensible to different logics and theories of agency
u Generate resultant conversation policies
– Off-line design simplifies agent development and reduces burden of
inference for agents at runtime
– Policies mediate interaction, helping increase interoperability and
robustness in heterogeneous environments
 Java Security and Mobility
u Java is currently the most popular and arguably the most
security-conscious mainstream language for agent
development
u Its cross-platform nature makes it well-suited for
heterogeneous environments
u However Java 1.0-1.1 failed to address many of the
challenges posed by agent software
– All or nothing philosophy in “sandbox”
– Lack of fine-grained resource control
– Security policy implementation requires writing your own security
manager
– Applet mechanisms are insufficient for autonomous agent mobility
 New Developments in Java
Security and Mobility
u Mechanisms for increasing configurability, extensibility,
and fine-grained access control are under development at
Sun Microsystems
u Java 1.2 enhancements
– Applets and applications on equivalent security footings
– Finer-grained configurability and better resource control
– Specification of much of the security policy via an external policy
file, thus separating policy from mechanism
u These new developments provide an initial foundation for
support of agent-unique requirements
 Security Design Tool (SDT)
u Accelerate incorporation of required agent security and mobility
features into the Java platform
– Foundation of new Java security model + changes to Java VM
– Work with vendors, developers, standards organizations
u Issues for Java platform enhancement and SDT development
– Agent authentication and PKI management
– Secure communication
– Enhanced configurability and resource management
• Denial of service issues: CPU, disk, memory, display
• Load balancing and grid “resource dial”
– Support for secure agent mobility
u SDT Benefits
– Configurable “starter set” of agent security policies
– Interoperability among different agent frameworks (grid “security dial”?)
– Faster creation of robust agents by non-experts
Agent “Scram” Capabilities for
Anytime Mobility
 Anytime Mobility
u Telescript provided completely transparent agent mobility
u Current Java-based agent systems do not
– Agent system code runs inside the VM; no access to execution state
u Advantages of transparent agent mobility
– Agent code need not be structured with many entry points
– Allows the agent system (as well as the agents themselves) to move
agents between hosts
– May be transparent to the agent (may require additional redirection
of agent resources)
– Supports load balancing of long running agents in the grid
u Requires modifications to the Java VM
Airplane Troubleshooting Evolution
"PAPER" BASED REFERENCE

(Standalone / Linearly Organized / Org Driven Tools (Stovepipes) / Unsync Revs / Not @ Jobsite / Rev Cycle  2 Mo.)

Today's Environment
(Not Process Oriented) DDG FIM AMM CLG Troubleshooting IPC
CBT
•  Variable Fault
  Download Tools
•  ACARS Reporting
•  FRM
•  Anecdotal
•  BITE

COMPUTER BASED REFERENCE

(Relevant Standalone Ref Data / Hyperlinked / Org Driven (StovePipe) / Semi­Sync Revs / @ Jobsite / Rev Cycle  2 Mo.)

PMA Prototype Dispatch Fault Airplane Illustrated

(Bridge to Process Oriented)
Deviation Isolation Maintenance Parts
•  Variable Fault Guide Manual Manual Catalog
  Download Tools (DDG) (FIM) (AMM) (IPC)
•  ACARS Reporting
•  FRM
•  Anecdotal Fly or Fix Troubleshooting Remove/Replace Parts Where &
•  BITE Test/Restore When Needed

INTELLIGENT PERFORMANCE SUPPORT & REFERENCE TOOLS
(Process Based / Hyperlinked / Intelligent Agents / Multimedia / Seamless Fault Det ­ Fly or Fix Res / Rev  Cycle Š 2 Mo.)

Vision (Process Oriented) Update Airline
In Flight Fault Personnel Fault Isolation Fly / Fix Update Data Enterprise Data
Process Steps Detection & Downlink Readiness Return to Service "Documents" System

Electronic QRH & Component Location Agent Assisted Multimedia (JIT) Feedback to the Fault Feedback to Airline

Approach ACARS Multimedia Training Trouble Shooting Agent Assisted Fix System Enterprise System
Enterprise Data R&R, Test &
Return to Service
Underlying New Component Integration Intelligent Agents Independent Links Wireless & Wearable Media Servers
Technologies Required Architecture Computing
11.130.6 Evolution
 Agent Roles in Technical Information
u Agent-Assisted Document
Construction
At the user-interface, agents work in
conjunction with compound
document and web browser
frameworks and document
management tools to select the right
data, assemble the needed
components, and present the
information in the most appropriate
way for a specific user and situation.
A
u Agent-Assisted Software
Integration
Behind the scenes, agents take A A

advantage of distributed object

management, database, workflow,
messaging, transaction, web, and
networking capabilities to discover, A A
link, manage, and securely access
the appropriate data and services.
 Aviation Extranet Goals
“By the turn of the century, airlines will be able to dynamically reconfigure their flight operations
for improved safety and more efficient transportation for the traveling public”

◆ Develop middleware components to integrate and extend the

capabilities of aviation legacy systems on a secure extranet to support:
– Real-time aircraft and airport situational awareness and scheduling and planning functions
– Maintenance and operations procedures enhancements
– Feedback data mechanisms to design/manufacturing models and simulators

◆ Develop Extranet Global Information Services

– Intelligent agents
– Metadatabases and Data Warehouses

◆ Conduct advanced research in decision support tools for the Aviation

Community
Aviation Extranet Goals (cont.)

Data Data Decision

Meta-Dbases Warehouses Mining Support Extranet
Systems Network
Intelligent Agents Hardware
CORBA Components
Legacy Data Systems
 Aviation Extranet Middleware Architecture
Industry Data
Industry Sources
Data Sources
Maintenance/
Industry Data Ancillary
Sources Meta­Dbases

Domain
Real­Time Ops Service Design/
Doman Stations Manufacturing
Meta­Dbases Service Meta­Dbases
Stations Domain
Service
Stations

Intelligent Web Servers
Regulations/
Documentation Domain
Meta­Dbases Service
Stations

Industry Data
Sources
CORBA  Interfaces Intelligent Agents Web Browser
 ● Authenticate Once
Extranet Security ●
●
Permission­Based Access
Encryptable Communication
Boeing 
Authenticate 
Web  (Reverse Proxy) 
Certificate 
Check 
DB  Server  & 
Certificate 
Check  Agent 

User 
Client 
CORBA  Certificate  A2A (over IIOP, TCP/IP, COM) 
Server  Check  Agent 
HTTP 
Agent 
IIOP 
Agent 
Data Access 

Agent 
Agent  Certificate 
Check 
Airline  Agent 
Certificate 
Gov't 
Check 

DB 
DB  CORBA  Agent 

DB 
Server 
CORBA 
CORBA  Server 
DB 
Server  DB 
 Agent-Based Framework for
Information Access
User User
Agent Agent

Matchmaker Matchmaker
Information Information
Agent Agent
Broker Broker
Agent Agent
Metadata/ Metadata/
Ontology Ontology
Agent Agent
Information Information Information
Service Service Service
Agent Agent Agent

* Matchmaker is connected to
Information Sources almost every agent