UNIT 4

SOFTWARE AGENTS

4.1 DEFINITION

Agent architectures, like software architectures, are formally a description of the

elements from which a system is built and the manner in which they communicate. Further,
these elements can be defined from patterns with specific constraints.

1. A number of common architectures exist that go by the names pipe-and filter or

layered architecture.

2. These define the interconnections between components.

3. Pipe-and-Filter defines a model where data is moved through a set of one or

more objects that perform a transformation.

4. Layered simply means that the system is comprised of a set of layers that provide
a specific set of logical functionality and that connectivity is commonly restricted
to the layers contiguous to one another.

4.2 TYPES OF ARCHITECTURES

Based on the goals of the agent application, a variety of agent architectures exist to help.
This section will introduce some of the major architecture types and applications for which
they can be used.

1. Reactive architectures
2. Deliberative architectures
3. Blackboard architectures
4. Belief-desire-intention (BDI) architecture
5. Hybrid architectures
6. Mobile architectures

1. REACTIVE ARCHITECTURES

1. A reactive architecture is the simplest architecture for agents.

2. In this architecture, agent behaviors are simply a mapping between stimulus and
response.

3. The agent has no decision-making skills, only reactions to the environment in which it
exists.
4. The agent simply reads the environment and then maps the state of the environment to
one or more actions. Given the environment, more than one action may be appropriate,
and therefore the agent must choose.

5. The advantage of reactive architectures is that they are extremely fast.

6. This kind of architecture can be implemented easily in hardware, or fast in software

lookup.

7. The disadvantage of reactive architectures is that they apply only to simple

environments.

8. Sequences of actions require the presence of state, which is not encoded into the
mapping function.

2. DELIBERATIVE ARCHITECTURES

1. A deliberative architecture, as the name implies, is one that includes some deliberation
over the action to perform given the current set of inputs.

2. Instead of mapping the sensors directly to the actuators, the deliberative architecture
considers the sensors, state, prior results of given actions, and other information in order
to select the best action to perform.

3. The mechanism for action selection as is undefined. This is because it could be a variety
of mechanisms including a production system, neural network, or any other
intelligent algorithm.

4. The advantage of the deliberative architecture is that it can be used to solve much more
complex problems than the reactive architecture.

5. It can perform planning, and perform sequences of actions to achieve a goal.

6. The disadvantage is that it is slower than the reactive architecture due to the deliberation
for the action to select.

Figure 4.1 Reactive architecture defines a simple agent

 Figure 4.2 A deliberative agent architecture considers its actions

3. BLACKBOARD ARCHITECTURES

1. The blackboard architecture is a very common architecture that is also very interesting.

2. The first blackboard architecture was HEARSAY-II, which was a speech understanding
system. This architecture operates around a global work area call the blackboard.

3. The blackboard is a common work area for a number of agents that work cooperatively
to solve a given problem.

4. The blackboard therefore contains information about the environment, but also
intermediate work results by the cooperative agents.

5. In this example, two separate agents are used to sample the environment through the
available sensors (the sensor agent) and also through the available actuators (action
agent).

6. The blackboard contains the current state of the environment that is constantly updated
by the sensor agent, and when an action can be performed (as specified in the
blackboard), the action agent translates this action into control of the actuators.

7. The control of the agent system is provided by one or more reasoning agents.

8. These agents work together to achieve the goals, which would also be contained in the
blackboard.

9. In this example, the first reasoning agent could implement the goal definition behaviors,
where the second reasoning agent could implement the planning portion (to translate
goals into sequences of actions).

10. Since the blackboard is a common work area, coordination must be provided such that
agents don’t step over one another.
11. For this reason, agents are scheduled based on their need. For example, agents can
monitor the blackboard, and as information is added, they can request the ability to
operate.

12. The scheduler can then identify which agents desire to operate on the blackboard, and
then invoke them accordingly.

13. The blackboard architecture, with its globally available work area, is easily
implemented with a multi-threading system.

14. Each agent becomes one or more system threads. From this perspective, the blackboard
architecture is very common for agent and non-agent systems.

Figure 4.3 The blackboard architecture supports multi-agent problem solving

4. BELIEF-DESIRE-INTENTION (BDI) ARCHITECTURE

1. BDI, which stands for Belief-Desire-Intention, is an architecture that follows the theory
of human reasoning as defined by Michael Bratman.

2. Belief represents the view of the world by the agent (what it believes to be the state of
the environment in which it exists). Desires are the goals that define the motivation of
the agent (what it wants to achieve).

3. The agent may have numerous desires, which must be consistent. Finally, Intentions
specify that the agent uses the Beliefs and Desires in order to choose one or more actions
in order to meet the desires.

4. As we described above, the BDI architecture defines the basic architecture of any
deliberative agent. It stores a representation of the state of the environment (beliefs),
maintains a set of goals (desires), and finally, an intentional element that maps desires
to beliefs (to provide one or more actions that modify the state of the environment based
on the agent’s needs).
 Figure 4.4 The BDI architecture desires to model mental attributes

5. HYBRID ARCHITECTURES

1. As is the case in traditional software architecture, most architectures are hybrids.

2. For example, the architecture of a network stack is made up of a pipe-and-filter

architecture and a layered architecture.

3. This same stack also shares some elements of a blackboard architecture, as there are
global elements that are visible and used by each component of the architecture.

4. The same is true for agent architectures. Based on the needs of the agent system,
different architectural elements can be chosen to meet those needs.

6. MOBILE ARCHITECTURES

1. The final architectural pattern that we will discuss is the mobile agent architecture.

2. This architectural pattern introduces the ability for agents to migrate themselves
between hosts. The agent architecture includes the mobility element, which allows an
agent to migrate from one host to another.

3. An agent can migrate to any host that implements the mobile framework.

4. The mobile agent framework provides a protocol that permits communication between
hosts for agent migration.

5. This framework also requires some kind of authentication and security, to avoid a
mobile agent framework from becoming a conduit for viruses. Also implicit in the
mobile agent framework is a means for discovery.
6. For example, which hosts are available for migration, and what services do they
provide? Communication is also implicit, as agents can communicate with one another
on a host, or across hosts in preparation for migration.

7. The mobile agent architecture is advantageous as it supports the development of

intelligent distributed systems. But a distributed system that is dynamic, and whose
configuration and loading is defined by the agents themselves.

Figure 4.5 The mobile agent framework supports agent mobility

7. ARCHITECTURE DESCRIPTIONS

1. Subsumption Architecture (Reactive Architecture)

2. Behavior Networks (Reactive Architecture)
3. ATLANTIS (Deliberative Architecture)
4. Homer (Deliberative Arch)
5. BB1 (Blackboard)
6. Open Agent Architecture (Blackboard)
7. Procedural Reasoning System (BDI)
8. Aglets (Mobile)
9. Messengers (Mobile)
10. Soar (Hybrid)

➢ SUBSUMPTION ARCHITECTURE (REACTIVE ARCHITECTURE)

1. The Subsumption architecture, originated by Rodney Brooks in the late 1980s, was
created out of research in behavior-based robotics.

2. The fundamental idea behind subsumption is that intelligent behavior can be created
through a collection of simple behavior modules.
3. These behavior modules are collected into layers. At the bottom are behaviors that are
reflexive in nature, and at the top, behaviors that are more complex. Consider the
abstract model shown in Figure.

4. At the bottom (level 0) exist the reflexive behaviors (such as obstacle avoidance). If
these behaviors are required, then level 0 consumes the inputs and provides an action at
the output. But no obstacles exist, so the next layer up is permitted to subsume control.

5. At each level, a set of behaviors with different goals compete for control based on the
state of the environment.

6. To support this capability levels can be inhibited (in other words, their outputs are
disabled). Levels can also be suppressed such that sensor inputs are routed to higher
layers. As shown in Figure.

7. Subsumption is a parallel and distributed architecture for managing sensors and

actuators. The basic premise is that we begin with a simple set of behaviors, and once
we’ve succeeded there, we extend with additional levels and higher- level behaviors.

8. For example, we begin with obstacle avoidance and then extend for object seeking.
From this perspective, the architecture takes a more evolutionary design approach.

9. Subsumption does have its problems. It is simple, but it turns out not to be extremely
extensible. As new layers are added, the layers tend to interfere with one another, and
then the problem becomes how to layer the behaviors such that each has the opportunity
to control when the time is right.

10. Subsumption is also reactive in nature, meaning that in the end, the architecture still
simply maps inputs to behaviors (no planning occurs, for example). What subsumption
does provide is a means to choose which behavior for a given environment.

Figure 4.6 Architectural view of the subsumption architecture

➢ BEHAVIOR NETWORKS (REACTIVE ARCHITECTURE)

1. Behavior networks, created by Pattie Maes in the late 1980s, is another reactive
architecture that is distributed in nature. Behavior networks attempt to answ er the
question, which action is best suited for a given situation.

2. As the name implies, behavior networks are networks of behaviors that include
activation links and inhibition links.

3. An example behavior network for a game agent is shown in Figure. As shown in the
legend, behaviors are rectangles and define the actions that the agent may take (attack,
explore, reload, etc.).

4. The ovals specify the preconditions for actions to be selected, which are inputs from the
environment.

5. Preconditions connect to behaviors through activation links (they promote the behavior
to be performed) or inhibition links (that inhibit the behavior from being performed).

6. The environment is sampled, and then the behavior for the agent is selected based on
the current state of the environment. The first thing to note is the activation and
inhibition links. For example, when the agent’s health is low, attack and exploration are
inhibited, leaving the agent to find the nearest shelter. Also, while exploring, the agent
may come across medkits or ammunition.

7. If a medkit or ammunition is found, it’s used. Maes’ algorithm referred to competence

modules, which included preconditions (that must be fulfilled before the module can
activate), actions to be performed, as well as a level of activation.

8. The activation level is a threshold that is used to determine when a competence module
may activate.

9. The algorithm also includes decay, such that activiations dissipate over time. Like the
subsumption architecture, behavior networks are instances of Behavior-Based Systems
(BBS). The primitive actions produced by these systems are all behaviors, based on the
state of the environment.

10. Behavior networks are not without problems. Being reactive, the architecture does not
support planning or higher- level behaviors. The architecture can also suffer when
behaviors are highly inter-dependent. With many competing goals, the behavior
modules can grow dramatically in order to realize the intended behaviors. But for
simpler architecture, such as the FPS game agent in Figure 4.7, this algorithm is ideal.
 Figure 4.7 Behavior network for a simple game agent

➢ ATLANTIS (Deliberative Architecture)

1. The goal of ATLANTIS (A Three-Layer Architecture for Navigating Through Intricate

Situations), was to create a robot that could navigate through dynamic and imperfect
environments in pursuit of explicitly stated high-level goals.

2. ATLANTIS was to prove that a goal-oriented robot could be built from a hybrid
architecture of lower-level reactive behaviors and higher- level deliberative behaviors.

Figure 4.8 ATLANTIS Architecture

3. Where the subsumption architecture allows layers to subsume control, ATLANTIS
operates on the assumption that these behaviors are not exclusive of one another. The
lowest layer can operate in a reactive fashion to the immediate needs of the
environment, while the uppermost layer can support planning and more goal-oriented
behaviors.

4. In ATLANTIS, control is performed from the bottom- up. At the lowest level (the
control layer) are the reactive behaviors.

5. These primitive level actions are capable of being executed first, based on the state of
the environment. At the next layer is the sequencing layer. This layer is responsible for
executing plans created by the deliberative layer.

6. The deliberative layer maintains an internal model of the environment and creates plans
to satisfy goals.

7. The sequencing layer may or may not complete the plan, based on the state of the
environment. This leaves the deliberation layer to perform the computationally
expensive tasks. This is another place that the architecture is a hybrid.

8. The lower- level behavior-based methods (in the controller layer) are integrated with
higher- level classical AI mechanisms (in the deliberative layer). Interestingly, the
deliberative layer does not control the sequencing layer, but instead simply advises on
sequences of actions that it can perform.

9. The advantage of this architecture is that the low- level reactive layer and higher- level
intentional layers are asynchronous. This means that while deliberative plans are under
construction, the agent is not susceptible to the dynamic environment. This is because
even though planning can take time at the deliberative layer, the controller can deal with
random events in the environment.

➢ HOMER (DELIBERATIVE ARCH)

1. Homer is another interesting deliberative architecture that is both modular and

integrated. Homer was created by Vere and Bickmore in 1990 as a deliberative
architecture with some very distinct differences to other architectures.

2. At the core of the Homer architecture is a memory that is divided into two parts. The
first part contains general knowledge (such as knowledge about the environment). The
second part is called episodic knowledge, which is used to record experiences in the
environment (perceptions and actions taken).

3. The natural language processor accepts human input via a keyboard, and parses and
responds using a sentence generator. The temporal planner creates dynamic plans to
satisfy predefined goals, and is capable of replanning if the environment requires.
4. The architecture also includes a plan executor (or interpreter), which is used to execute
the plan at the actuators. The architecture also included a variety of monitor processes.
The basic idea behind Homer was an architecture for general intelligence.

5. The keyboard would allow regular English language input, and a terminal would display
generated English language sentences. The user could therefore communicate with
Homer to specify goals and receive feedback via the terminal.

6. Homer could log perceptions of the world, with timestamps, to allow dialogue with the
user and rational answers to questions. Reflective (monitor) processes allow Homer to
add or remove knowledge from the episodic memory.

7. Homer is an interesting architecture implementing a number of interesting ideas, from

natural language processing to planning and reasoning. One issue found in Homer is
that when the episodic memory grows large, it tends to slow down the overall operation
of the agent.

Figure 4.9 Homer Architecture

➢ BB1 (BLACKBOARD)

1. BB1 is a domain- independent blackboard architecture for AI systems created by

Barbara Hayes- Roth. The architecture supports control over problem solving as well
as explaining its actions. The architecture is also able to learn new domain knowledge.
2. BB1 includes two blackboards; a domain blackboard which acts as the global database
and a control blackboard, which is used for generating a solution to the given control
problem.

3. The key behind BB1 is its ability to incrementally plan. Instead of defining a complete
plan for a given goal, and then executing that plan, BB1 dynamically develops the plan
and adapts to the changes in the environment. This is key for dynamic environments,
where unanticipated changes can lead to brittle plans that eventually fail.

➢ PROCEDURAL REASONING SYSTEM (BDI)

1. The Procedural Reasoning System (PRS) is a general-purpose architecture that’s ideal

for reasoning environments where actions can be defined by predetermined procedures
(action sequences).

2. PRS is also a BDI architecture, mimicking the theory on human reasoning. PRS
integrates both reactive and goal-directed deliberative processing in a distributed
architecture.

3. The architecture is able to build a world-model of the environment (beliefs) through

interacting with environment sensors.

4. Actions can also be taken through an intentions module. At the core is an interpreter (or
reasoner) which selects a goal to meet (given the current set of beliefs) and then retrieves
a plan to execute to achieve that goal. PRS iteratively tests the assumptions of the plan
during its execution. This means that it can operate in dynamic environments where
classical planners are doomed to fail.

5. Plans in PRS (also called knowledge areas) are predefined for the actions that are
possible in the environment. This simplifies the architecture because it isn’t required to
generate plans, only select them based on the environment and the goals that must be
met.

6. While planning is more about selection than search or generation, the interpreter ensures
that changes to the environment do not result in inconsistencies in the plan. Instead, a
new plan is selected to achieve the specific goals.

7. PRS is a useful architecture when all necessary operations can be predefined. It’s also
very efficient due to lack of plan generation. This makes PRS an ideal agent architecture
for building agents such as those to control mobile robots.

➢ AGLETS (MOBILE)

1. Aglets is a mobile agent framework designed by IBM Tokyo in the 1990s. Aglets is
based on the Java programming language, as it is well suited for a mobile agents
framework. First, the applications are portable to any system (both homogeneous and
 heterogeneous) that is capable of running a Java Virtual Machine (JVM). Second, a
JVM is an ideal platform for migration services.

2. Java supports serialization, which is the aggregation of a Java application’s program

and data into a single object that is restart able.

3. In this case, the Java application is restarted on a new JVM. Java also provides a secure
environment (sandbox) to ensure that a mobile agent framework doesn’t become a virus
distribution system. The Aglets framework is shown in Figure 4.9. At the bottom of the
framework is the JVM (the virtual machine that interprets the Java byte codes). The
agent runtime environment and mobility protocol are next. The mobility protocol, called
Aglet Transport Protocol (or ATP), provides the means to serialize agents and then
transport them to a host previously defined by the agent.

4. The agent API is at the top of the stack, which in usual Java fashion, provides a number
of API classes that focus on agent operation. Finally, there are the various agents that
operate on the framework.

5. The agent API and runtime environment provide a number of services that are central
to a mobile agent framework. Some of the more important functions are agent
management, communication, and security. Agents must be able to register themselves
on a given host to enable communication from outside agents.

6. In order to support communication, security features must be implemented to ensure

that the agent has the authority to execute on the framework.

7. Aglets provides a number of necessary characteristics for a mobile agent framework,

including mobility, communication, security, and confidentiality. Aglets provide weak
migration, in that the agents can only migrate at arbitrary points within the code (such
as with the dispatch method).

➢ MESSENGERS (MOBILE)

1. Messengers is a runtime environment that provides a form of process migration (mobile

agency).

2. One distinct strength of the messengers environment is that it supports strong migration,
or the ability to migrate at arbitrary points within the mobile application.

3. The messengers environment provides the hop statement which defines when and where
to migrate to a new destination.

4. After migration is complete, the messengers agent restarts in the application at the point
after the previous hop statement. The end result is that the application moves to the data,
rather than using a messaging protocol to move the data to the agent.
5. There are obvious advantages to this when the data set is large and the migration links
are slow. The messengers model provides what the authors call Navigational
Programming, and also Distributed Sequential Computing (DSC).

6. What makes these concepts interesting is that they support the common model of
programming that is identical to the traditional flow of sequential programs. This makes
them easier to develop and understand.

7. Let’s now look at an example of DSC using the messengers environment. Listing 11.5
provides a simple program. Consider an application where on a series of hosts, we
manipulate large matrices which are held in their memory.

➢ SOAR (HYBRID)

1. Soar, which originally was an acronym for State-Operator-And-Result, is a symbolic

cognitive architecture.

2. Soar provides a model of cognition along with an implementation of that model for
building general-purpose AI systems.

3. The idea behind Soar is from Newell’s unified theories of cognition. Soar is one of the
most widely used architectures, from research into aspects of human behavior to the
design of game agents for first person- shooter games.

4. The goal of the Soar architecture is to build systems that embody general intelligence.
While Soar includes many elements that support this goal (for example, representing
knowledge using procedural, episodic, and declarative forms), but Soar lacks some
important aspects. These include episodic memories and also a model for emotion.
Soar’s underlying problem-solving mechanism is based on a production system (expert
system).

5. Behavior is encoded in rules similar to the if-then form. Solving problems in Soar can
be most simply described as problem space search (to a goal node). If this model of
problem solving fails, other methods are used, such as hill climbing.

6. When a solution is found, Soar uses a method called chunking to learn a new rule based
on this discovery. If the agent encounters the problem again, it can use the rule to select
the action to take instead of performing problem solving again.

4.3 AGENT COMMUNICATION

In the domain of multi-agent systems, communication is an important characteristic to

support both coordination and the transfer of information. Agents also require the ability
to communicate actions or plans. But how the communication takes place is a function
of its purpose.
1. Agents communicate in order to achieve better the goals of themselves or of the society/
system in which they exist.

2. Communication can enable the agents to coordinate their actions and behavior, resulting
in systems that are more coherent.

3. Coordination is a property of a system of agents performing some activity in a shared

environment.

4. The degree of coordination is the extent to which they avoid extraneous activity by
reducing resource contention, avoiding live lock and deadlock, and maintaining
applicable safety conditions.

5. Cooperation is coordination among non-antagonistic agents, while negotiation is

coordination among competitive or simply self-interested agents.

6. Typically, to cooperate successfully, each agent must maintain a model of the other
agents, and also develop a model of future interactions. This presupposes sociability
Coherence is how well a system behaves as a unit. A problem for a multiagent system
is how it can maintain global coherence without explicit global control. In this case, the
agents must be able on their own to determine goals they share with other agents,
determine common tasks, avoid unnecessary conflicts, and pool knowledge and
evidence. It is helpful if there is some form of organization among the agents.

Dimensions of Meaning

There are three aspects to the formal study of communication: syntax (how the symbols
of communication are structured), semantics (what the symbols denote), and pragmatics (how
the symbols are interpreted). Meaning is a combination of semantics and pragmatics. Agents
communicate in order to understand and be understood, so it is important to consider the
different dimensions of meaning that are associated with communication.

1. Descriptive vs. Prescriptive. Some messages describe phenomena, while others

prescribe behaviour. Descriptions are important for human comprehension, but are
difficult for agents to mimic. Appropriately, then, most agent communication languages
are designed for the exchange of information about activities and behaviour.

2. Personal vs. Conventional Meaning. An agent might have its own meaning for a
message, but this might differ from the meaning conventionally accepted by the other
agents with which the agent communicates. To the greatest extent possible, multiagent
systems should opt for conventional meanings, especially since these systems are
typically open environments in which new agents might be introduced at any time.

3. Subjective vs. Objective Meaning Similar to conventional meaning, where meaning is

determined external to an agent, a message often has an explicit effect on the
 environment, which can be perceived objectively. The effect might be different than
that understood internally, i.e., subjectively, by the sender or receiver of the message.

4. Speaker's vs. Hearer's vs. Society's Perspective Independent of the conventional or

objective meaning of a message, the message can be expressed according to the
viewpoint of the speaker or hearer or other observers.

5. Semantics vs. Pragmatics The pragmatics of a communication are concerned with how
the communicators use the communication. This includes considerations of the mental
states of the communicators and the environment in which they exist, considerations
that are external to the syntax and semantics of the communication.

6. Contextuality Messages cannot be understood in isolation, but must be interpreted in

terms of the mental states of the agents, the present state of the environment, and the
environment's history: how it arrived at its present state. Interpretations are directly
affected by previous messages and actions of the agents.

4.4 MESSAGE TYPES

1. It is important for agents of different capabilities to be able to communicate.

Communication must therefore be defined at several levels, with communication at the
lowest level used for communication with the least capable agent.

2. In order to be of interest to each other, the agents must be able to participate in a

dialogue. Their role in this dialogue may be either active, passive, or both, allowing
them to function as a master, slave, or peer, respectively.

3. In keeping with the above definition for and assumptions about an agent, we assume
that an agent can send and receive messages through a communication network.

4. The messages can be of several types, as defined next.

5. There are two basic message types: assertions and queries. Every agent, whether active
or passive, must have the ability to accept information. In its simplest form, this
information is communicated to the agent from an external source by means of an
assertion. In order to assume a passive role in a dialog, an agent must additionally be
 able to answer questions, i.e., it must be able to 1) accept a query from an external
source and 2) send a reply to the source by making an assertion. Note that from the
standpoint of the communication network, there is no distinction between an unsolicited
assertion and an assertion made in reply to a query.

6. In order to assume an active role in a dialog, an agent must be able to issue queries and
make assertions. With these capabilities, the agent then can potentially control another
agent by causing it to respond to the query or to accept the information asserted. This
means of control can be extended to the control of subagents, such as neural networks
and databases.

4.5 SPEECH ACTS

Spoken human communication is used as the model for communication among

computational agents. A popular basis for analyzing human communication is speech act theory
[1, 39]. Speech act theory views human natural language as actions, such as requests,
suggestions, commitments, and replies. For example, when you request something, you are not
simply making a statement, but creating the request itself. When a jury declares a defendant
guilty, there is an action taken: the defendant's social status is changed.

A speech act has three aspects:

1. Locution, the physical utterance by the speaker

2. Illocution, the intended meaning of the utterance by the speaker

3. Perlocution, the action that results from the locution. KQML (Knowledge Query and
Manipulation Language)

1. The KQML is an interesting example of communication from a number of facets. For

example, communication requires the ability to locate and engage a peer in a
conversation (communication layer).

2. A method for packaging the messages is then necessary (messaging layer), and finally
an internal format that represents the messages and is sufficiently expressive to convey
not only information but requests, responses, and plans (content layer).

3. In a network of KQML-speaking agents, there exists programs to support

communication. These consist of facilitators that can serve as name servers to KQML
components, and help find other agents that can satisfy a given agent’s request.

4. A KQML router supports the routing of messages and is a front-end to a specific KQML
agent. As KQML was originally written in Common LISP, it’s message representation
follows the LISP example (balanced parentheses).
5. A KQML message can be transferred to any particular transport (such as sockets) and
has a format that consists of a performative and a set of arguments for that performative.
The performative defines the speech act which defines the purpose of the message
(assertion, command, request, etc.).

6. The performative- name defines the particular message type to be communicated

(evaluate, ask- if, stream-about, reply, tell, deny, standby, advertise, etc.). The sender
and receiver define the unique names of the agents in the dialogue. The content is
information specific to the performative being performed.

7. This content is defined in a language (how to represent the content), and an ontology
that describes the vocabulary (and meaning) of the content. Finally, the agent can attach
a context which the response will contain (in-reply-to) in order to correlate the request
with the response. The structure of a KQML message.

(performative-name
: sender X
: receiver Y
: content Z
: language L
: ontology Y
: reply-with R
: in-reply-to Q
)

Let’s now look at an example conversation between two KQML agents. In thisexample,
an agent requests the current value of a temperature sensor in a system. The request is for the
temperature of TEMP_SENSOR_1A that’s sampled at the temperature-server agent. The
content is the request, defined in the prolog language. Our agent making the request is called
thermal-control-appl.

(ask-one
:sender thermal-control-appl
:receiver temperature-server
:languageprolog
:ontology CELSIUS-DEGREES
:content “temperature(TEMP_SENSOR_1A ?temperature)”
:reply-with request-102
)

Our agent would then receive a response from the temperature-server, defining the
temperature of the sensor of interest.
 (reply
:sender temperature-server
:receiver thermal-control-appl
:languageprolog
:ontology CELSIUS-DEGREES
:content “temperature(TEMP_SENSOR_1A 45.2)”
:in-reply-to request-102
)

Our agent would then receive a response from the temperature-server, defining the
temperature of the sensor of interest,

(reply
:sender temperature-server
:receiver thermal-control-appl
:languageprolog
:ontology CELSIUS-DEGREES
:content “temperature(TEMP_SENSOR_1A 45.2_”
:in-reply-to request1102
)

KQML is very rich in its ability to communicate information as well higher-level

request that address the communication layer. Table 4.1 provides a short list of some of the
other KQML per formatives.

4.6 KQML PERFORMATIVES

Table 4.1

Performatives Description
Evaluate Evaluate the content of the message
ask-one Request for the answer to the question
reply Communicate a reply to a question
stream-about Provide multiple response to a question
sorry Return an error (can’t respond)
tell Inform an agent of sentence
achieve A request of something to achieve by the receiver
advertise Advertise the ability to process a performative
subscribe Subscribe to changes of information
forward Route a message
 KQML is a useful language to communicate not only data, but the meaning of the data
(in terms of a language and ontology).

 KQML provides a rich set of capabilities that cover basic speed acts, and more complex
acts including data streaming and control of information transfer.

4.7 ACL (FIPA AGENT COMMUNICATION LANGUAGE)

1. Where KQML is a language defined in the context of a university, the FIPA ACL is a
consortium-based language for agent communication.

2. ACL simply means Agent Communication Language and it was standardized through
the Foundation for Intelligent Physical Agents consortium. As with KQML, ACL is a
speech- act language defined by a set of per formatives.

3. The FIPA, or Foundation for Intelligent Physical Agents, is a non-profit organization

that promotes the development of agent-based systems. It develops specifications to
maximize the portability of agent systems (including their ability to communicate using
the ACL).

4. The FIPA ACL is very similar to the KQML, even adopting the inner and outer content
layering for message construction (meaning and content).

5. The ACL also clarifies certain speech-acts, or performatives. For example,

communication primitives are called communicative acts, which are separate from the
performative acts.

6. The FIPA ACL also uses the Semantic Language, or SL, as the formal language to
define ACL semantics. This provides the means to support BDI themes (beliefs, desires,
intentions). In other words, SL allows the representation of persistent goals (intentions),
as well as propositions and objects. Each agent language has its use, and while both
have their differences, they can also be viewed as complementary.

XML

1. XML is the Extensible Markup Language and is an encoding that represents data and
meta- data (meaning of the data). It does this with a representation that includes tags
that encapsulate the data.

2. The tags explicitly define what the data represents. For example, consider the ask-one
request from KQML. This can be represented as XML as shown below:
1. There are some obvious similarities to XML and KQML. In KQML, the tags exist,
but use different syntax than is defined for XML. One significant difference is that
KQML permits the layering of tags.

2. Note here that the <msg> tag is the outer layer of the performative and its arguments.
XML is very flexible in its format and permits very complex arrangements of both
data and meta-data.

3. XML is used in a number of protocols, including XML-RPC (Remote Procedure Call)

and also SOAP (Simple Object Access Protocol). Each of these use the Hyper Text
Transport Protocol (HTTP) as its transport.

TRUST AND REPUTATION

It depends on the level we apply it:

1. User confidence
• Can we trust the user behind the agent?
– Is he/she a trustworthy source of some kind of knowledge? (e.g. an expert
in a field)
– Does he/she acts in the agent system (through his agents in a trustworthy
way?

2. Trust of users in agents

• Issues of autonomy: the more autonomy, less trust
• How to create trust?
– Reliability testing for agents
– Formal methods for open MAS
– Security and verifiability

3. Trust of agents in agents

• Reputation mechanisms
• Contracts
• Norms and Social Structures

What is Trust?

1. In closed environments, cooperation among agents is included as part of the designing

process:
2. The multi- agent system is usually built by a single developer or a single team of
developers and the chosen developers, option to reduce complexity is to ensure
cooperation among the agents they build including it as an important system
requirement.

3. Benevolence assumption: an agent AI requesting information or a certain service from

agent aj can be sure that such agent will answer him if AI has the capabilities and the
resources needed, otherwise aj will inform AI that it cannot perform the action
requested.

4. It can be said that in closed environments trust is implicit.

Trust can be computed as

1. A binary value (1=‘I do trust this agent’, 0=‘I don’t trust this agent’)

2. A set of qualitative values or a discrete set of numerical values (e g ‘trust always’ ‘trust
conditional to X’ ‘no trust’) e.g. always, X, trust ) (e.g. ‘2’, ‘1’, ‘0’, ‘-1’, ‘-2’)

3. A continuous numerical value (e.g. [-300..300])

4. A probability distribution

5. Degrees over underlying beliefs and intentions (cognitive approach)

HOW TO COMPUTE TRUST

1. Trust values can be externally defined

• by the system designer: the trust values are pre-defined
• by the human user: he can introduce his trust values about the humans behind the other
agents

2. Trust values can be inferred from some existing representation about the interrelations
between the agents
• Communication patterns, cooperation history logs, e-mails, webpage connectivity
mapping...

3. Trust values can be learnt from current and past experiences

• Increase trust value for agent AI if behaves properly with us
• Decrease trust value for agent AI if it fails/defects us
4. Trust values can be propagated or shared through a MAS
• Recommender systems, Reputation mechanisms.

TRUST AND REPUTATION

1. Most authors in literature make a mix between trust and reputation

2. Some authors make a distinction between them

3. Trust is an individual measure of confidence that a given agent has over other agent(s)

4. Reputation is a social measure of confidence that a group of agents or a society has over
agents or groups. Reputation is one mechanism to compute (individual) Trust

• I will trust more an agent that has good reputation

• My reputation clearly affects the amount of trust that others have towards me.

• Reputation can have a sanctioning role in social groups: a bad reputation can be very
costly to one’s future transactions.

5. Most authors combine (individual) Trust with some form of (social) Reputation in their
models

6. Recommender systems, Reputation mechanisms.

Figure 4.10 TRUST AND REPUTATION

 Direct experiences are the most relevant and reliable information source for individual
trust/reputation

1. Type 1: Experience based on direct interaction with the

2. Type 1: Experience with the partner
1. Used by almost all models
2. How to:
• trust value about that partner increases with good experiences,
• it decreases with bad ones
3. Problem: how to compute trust if there is no previous interaction?
3. Type 2: Experience based on observed interaction of other members
1. Used only in scenarios prepared for this.
2. How to: depends on what an agent can observe
• agents can access to the log of past interactions of other agents
• agents can access some feedback from agents about their past interactions (e.g., in
eBay)
3. Problem: one has to introduce some noise handling or
4. confidence level on this information
4. Prior-derived: agents bring with them prior beliefs about strangers
Used by some models to initialize trust/reputation values
How-to:
• designer or human user assigns prior values
• a uniform distribution for reputation priors is set

Give new agents the lowest possible reputation value there is no incentive to throw
away a cyber-identity when an agent’s reputation falls below a starting point.

• Assume neither good nor bad reputation for unknown agents.

• Avoid lowest reputation for new, valid agents as an obstacle for other agents to realize
that they are valid.
5. Group-derived:
• Models for groups can been extended to provide prior reputation estimates for agents
in social groups.
• Mapping between the initial individual reputation of a stranger and the group from
which he or she comes from.
• Problem: highly domain-dependent and model-dependent.
6. Propagated:
• Agent can attempt to estimate the stranger’s reputation based on information garnered
from others in the environment Also called word of mouth.
• Problem: The combination of the different reputation values tends to be an ad-hoc
solution with no social basis

TRUST AND REPUTATION MODELS

1. Not really for MAS, but can be applied to MAS

2. Idea: For serious life / business decisions, you want the
• opinion of a trusted e pert trusted expert
3. If an expert not personally known, then want to find a reference to one via a chain of
friends and colleagues
4. Referral-chain provides:
• Way to judge quality of expert's advice
• Reason for the expert to respond in a trustworthy manner
• Finding good referral-chains is slow, time-consuming, but vital business gurus on
“networking”
• Set of all possible referral-chains = a social network
5. Model integrates information from
• Official organizational charts (online)
• Personal web pages (+ crawling)
• External publication databases
• Internal technical document databases
6. Builds a social network based in referral chains
• Each node is a recommender agent
• Each node provides reputation values for specific areas
o E.g. Frieze is good in mathematics
• Searches in the referral network are
• made by areas
o E.g. browsing the network’s “mathematics” recommendation chains
7. Trust Model Overview
• 1-to-1 asymmetric trust relationships.
• Direct trust and recommender trust.
• Trust categories and trust values
• [-1,0,1,2,3,4].
8. Conditional transitivity.
Alice trusts Bob.&. Bob trusts Cathy
Alice trusts Cathy
Alice trusts.rec Bob.&. Bob says Bob trusts Cathy
Alice may trust Cathy
Alice trusts.rec Bob value X. &. Bob says Bob trusts Cathy value Y
 Alice may trust Cathy value f(X, Y)
9. Recommendation protocol
1. Alice ->Bob: RRQ(Eric)
2. Bob ->Cathy: RRQ(Eric)
3. Cathy -> Bob: Rec(Eric,3)
4. Bob ->Alice: Rec(Eric,3)

Figure 4.11 Procedure

12. Direct Trust:

■ ReGreT assumes that there is no difference between direct interaction and direct
observation in terms of reliability of the information. It talks about direct experiences.
■ The basic element to calculate a direct trust is the outcome.
13. Reputation Model: Witness reputation
a. First step to calculate a witness reputation is to identify the set of witnesses that will be
taken into account by the agent to perform the calculation.
b. Next step is to aggregate these values to obtain a single value for the witness reputation.
The importance of each piece of information in the final reputation value will be
proportional to the witness credibility.

14. Reputation Model: Witness reputation

a. Two methods to evaluate witness credibility:
i. ReGreT uses fuzzy rules to calculate how the structure of social relations influences the
credibility on the information. The antecedent of each rule is the type and degree of a
social relation (the edges in a sociogram) and the consequent is the credibility of the
witness from the point of view of that social relation.
ii The second method used in the ReGreT system to calculate the credibility of a witnessis
to evaluate the accuracy of previous pieces of information sent by that witness to the
agent. The agent is using the direct trust value to measure the truthfulness of the
information received from witnesses.

15. Reputation Model: Neighbourhood Reputation

a. Neighbourhood in a MAS is not related with the physical location of the agents but with
the links created through interaction.
b. The main idea is that the behaviour of these neighbours and the kind of relation they
have with the target agent can give some clues about the behaviour of the target agent.
c. To calculate a Neighbourhood Reputation the ReGreT system uses fuzzy rules.
i. The antecedents of these rules are one or several direct trusts associated to different
behavioural aspects and the relation between the target agent and the neighbour.
ii. The consequent is the value for a concrete reputation (that can be associated to the
same behavioural aspect of the trust values or not).

16. Reputation Model: System Reputation

a. To use the common knowledge about social groups and the role that the agent is playing
in the society as a mechanism to assign default reputations to the agents.
b. ReGreT assumes that the members of these groups have one or several observable
features that unambiguously identify their membership.
c. Each time an agent performs an action we consider that it is playing a single role.

17. System reputations are calculated using a table for each social group where the rows
are the roles the agent can play for that group, and the columns the behavioural aspects.

18. Reputation Model: Default Reputation

a. To the previous reputation types we have to add a fourth one, the reputation assigned
to a third party agent when there is no information at all: the default reputation.
b. Usually this will be a fixed value

19. Reputation Model: Combining reputations

a. Each reputation type has different characteristics and there are a lot of heuristics that
can be used to aggregate the four reputation values to obtain a single and representative
reputation value.
b. In ReGreT this heuristic is based on the default and calculated reliability assigned to
each type.

20. Main criticism to Trust and Reputation research:

a. Proliferation of ad-hoc models weakly grounded in social theory
b. No general, cross-domain model for reputation
c. Lack of integration between models

NEGOTIATION

1. A frequent form of interaction that occurs among agents with different goals is termed
negotiation.

2. Negotiation is a process by which a joint decision is reached by two or more agents,

each trying to reach an individual goal or objective. The agents first communicate their
positions, which might conflict, and then try to move towards agreement by making
concessions or searching for alternatives.

3. The major features of negotiation are (1) the language used by the participating agents,
(2) the protocol followed by the agents as they negotiate, and (3) the decision process
that each agent uses to determine its positions, concessions, and criteria for agreement.

4. Many groups have developed systems and techniques for negotiation. These can be
either environment-centered or agent-centered. Developers of environment-centered
techniques focus on the following problem: "How can the rules of the environment be
 designed so that the agents in it, regardless of their origin, capabilities, or intentions,
will interact productively and fairly?"

The resultant negotiation mechanism should ideally have the following attributes:

• Efficiency: the agents should not waste resources in coming to an agreement. Stability:
no agent should have an incentive to deviate from agreed- upon strategies.

• Simplicity: the negotiation mechanism should impose low computational and

bandwidth demands on the agents.

• Distribution: the mechanism should not require a central decision maker.

• Symmetry: the mechanism should not be biased against any agent for arbitrary or
inappropriate reasons.

5. An articulate and entertaining treatment of these concepts is found in [36]. In particular,

three types of environments have been identified: worth-oriented domains, state-
oriented domains, and task-oriented domains.

6. A task-oriented domain is one where agents have a set of tasks to achieve, all resources
needed to achieve the tasks are available, and the agents can achieve the tasks without
help or interference from each other. However, the agents can benefit by sharing some
of the tasks. An example is the "Internet downloading domain," where each agent is
given a list of documents that it must access over the Internet. There is a cost associated
with downloading, which each agent would like to minimize. If a document is common
to several agents, then they can save downloading cost by accessing the document once
and then sharing it.

7. The environment might provide the following simple negotiation mechanism and
constraints:
(1) each agent declares the documents it wants
(2) documents found to be common to two or more agents are assigned to agents
based on the toss of a coin,
(3) agents pay for the documents they download, and
(4) agents are granted access to the documents they download. as well as any in
their common sets. This mechanism is simple, symmetric, distributed, and
efficient (no document is downloaded twice). To determine stability, the agents'
strategies must be considered.

8. An optimal strategy is for an agent to declare the true set of documents that it needs,
regardless of what strategy the other agents adopt or the documents they need. Because
there is no incentive for an agent to diverge from this strategy, it is stable.
9. For the first approach, speech-act classifiers together with a possible world semantics
are used to formalize negotiation protocols and their components. This clarifies the
conditions of satisfaction for different kinds of messages. To provide a flavor of this
approach, we show in the following example how the commitments that an agent might
make as part of a negotiation are formalized [21]:

10. This rule states that an agent forms and maintains its commitment to achieve ø
individually iff (1) it has not precommitted itself to another agent to adopt and achieve
ø, (2) it has a goal to achieve ø individually, and (3) it is willing to achieve ø
individually. The chapter on "Formal Methods in DAI" provides more information on
such descriptions.

11. The second approach is based on an assumption that the agents are economically
rational. Further, the set of agents must be small, they must have a common language
and common problem abstraction, and they must reach a common solution. Under these
assumptions, Rosenschein and Zlotkin [37] developed a unified negotiation protocol.
Agents that follow this protocol create a deal, that is, a joint plan between the agents
that would satisfy all of their goals. The utility of a deal for an agent is the amount he
is willing to pay minus the cost of the deal. Each agent wants to maximize its own utility.

The agents discuss a negotiation set, which is the set of all deals that have a positive utility for
every agent.

In formal terms, a task-oriented domain under this approach becomes a tuple <T, A, c>

where T is the set of tasks, A is the set of agents, and c(X) is a monotonic function for the cost
of executing the tasks X. A deal is a redistribution of tasks. The utility of deal d for agent k is
Uk(d) = c(Tk) - c(dk)

The conflict deal D occurs when the agents cannot reach a deal. A deal d is individually rational
if d > D. Deal d is pareto optimal if there is no deal d' > d. The set of all deals that are
individually rational and pareto optimal is the negotiation set, NS. There are three possible
situations:

1. conflict: the negotiation set is empty

2. compromise: agents prefer to be alone, but since they are not, they will agree to a
negotiated deal
 3. cooperative: all deals in the negotiation set are preferred by both agents over achieving
their goals alone.

When there is a conflict, then the agents will not benefit by negotiating—they are better
off acting alone. Alternatively, they can "flip a coin" to decide which agent gets to satisfy its
goals.

Negotiation is the best alternative in the other two cases.

Since the agents have some execution autonomy, they can in principle deceive or
mislead each other. Therefore, an interesting research problem is to develop protocols or
societies in which the effects of deception and misinformation can be constrained. Another
aspect of the research problem is to develop protocols under which it is rational for agents to
be honest with each other. The connections of the economic approaches with human-oriented
negotiation and argumentation have not yet been fully worked out.

4. 7 BARGAINING

Link: http://www.cse.iitd.ernet.in/~rahul/cs905/lecture15/index.html (Refer this link

for easy understanding) A bargaining problem is defined as a pair (S,d). A bargaining solution
is a function f that maps every bargaining problem (S,d) to an outcome in S, i.e.,

f : (S,d) → S

Thus the solution to a bargaining problem is a pair in R2. It gives the values of the game
to the two players and is generated through the function called bargaining function. Bargaining
function maps the set of possible outcomes to the set of acceptable ones.

Bargaining Solution

In a transaction when the seller and the buyer value a product differently, a surplus is
created. A bargaining solution is then a way in which buyers and sellers agree to divide the
surplus. For example, consider a house made by a builder A. It costed him Rs.10 Lacs. A
potential buyer is interested in the house and values it at Rs.20 Lacs. This transaction can
generate a surplus of Rs.10 Lacs. The builder and the buyer now need to trade at a price. The
buyer knows that the cost is less than 20 Lacs and the seller knows that the value is greater than
10 Lacs. The two of them need to agree at a price. Both try to maximize their surplus. Buyer
would want to buy it for 10 Lacs, while the seller would like to sell it for 20 Lacs. They bargain
on the price, and either trade or dismiss. Trade would result in the generation of surplus,
whereas no surplus is created in case of no-trade. Bargaining Solution provides an acceptable
way to divide the surplus among the two parties. Formally, a Bargaining Solution is defined as,
F : (X,d) → S,

where X  R2 and S,d R2. X represents the utilities of the players in the set of possible
bargaining agreements. d represents the point of disagreement. In the above example, price 

[10,20], bargaining set is simply x + y  10, x  0, y  0. A point (x,y) in the bargaining set
represents the case, when seller gets a surplus of x, and buyer gets a surplus of y, i.e. seller sells
the house at 10 + x and the buyer pays 20 − y.

1. the set of payoff allocations that are jointly feasible for the two players in the process
of negotiation or arbitration, and

2. the payoffs they would expect if negotiation or arbitration were to fail to reach a
settlement.

Based on these assumptions, Nash generated a list of axioms that a reasonable solution
ought to satisfy. These axioms are as follows:

Axiom 1 (Individual Rationality) This axiom asserts that the bargaining solution should
give neither player less than what it would get from disagree ment, i.e., f(S,d) ≥ d.

Axiom 2 (Symmetry) As per this axiom, the solution should be independent of the
names of the players, i.e., who is named a and who is named b. This means that when the
players’ utility functions and their disagreement utilities are the same, they receive equal shares.
So any symmetries in the final payoff should only be due to the differences in their utility
functions or their disagreement outcomes.

Axiom 3 (Strong Efficiency) This axiom asserts that the bargaining solution should be
feasible and Pareto optimal.

Axiom 4 (Invariance) According to this axiom, the solution should not change as a
result of linear changes to the utility of either player. So, for example, if a player’s utility
function is multiplied by 2, this should not change the solution. Only the player will value what
it gets twice as much.

Axiom 5 (Independence of Irrelevant Alternatives) This axiom asserts that eliminating

feasible alternatives (other than the disagreement point) that would not have been chosen
should not affect the solution, i.e., for any closed convex set

Nash proved that the bargaining solution that satisfies the above five axioms is given

by:

NON-COOPERATIVE MODELS OF SINGLE-ISSUE NEGOTATION

Table 4.2

Cooperative Game Non Cooperative Game

 The players are allowed to communicate Each player independently chooses its
before choosing their strategies and playing strategy.
the game.
The basic modeling unit is the group The basic modeling unit is the individual.
Players can enforce cooperation in the group The cooperation between individuals is self-
through a third party. enforcing.

4.8 GAME-THEORETIC APPROACHES FOR MULTI-ISSUE NEGOTIATION

The following are the four key procedures for bargaining over multiple issue:

1. Global bargaining: Here, the bargaining agents directly tackle the global problem in
which all the issues are addressed at once. In the context of non-cooperative theory, the
global bargaining procedure is also called the package deal procedure. In this procedure,
an offer from one agent to the other would specify how each one of the issues is to be
resolved.

2. Independent/separate bargaining: Here negotiations over the individual issues are

totally separate and independent, with each having no effect on the other. This would
be the case if each of the two parties employed m agents (for negotiating over m issues),
with each agent in charge of negotiating one issue. For example, in negotiations
between two countries, each issue may be resolved by representatives from the
countries who care only about their individual issue.

3. Sequential bargaining with independent implementation: Here the two parties consider
one issue at a time. For instance, they may negotiate over the first issue, and after
reaching an agreement on it, move on to negotiate the second, and so on. Here, the
parties may not negotiate an issue until the previous one is resolved. There are several
forms of the sequential procedure. These are defined in terms of the agenda and the
implementation rule. For sequential bargaining, the agenda3 specifies the order in
which the issues will be bargained. The implementation rule specifies when an
agreement on an individual issue goes into effect. There are two implementation rules:
the rule of independent implementation and the rule of simultaneous implementation.

4. Sequential bargaining with simultaneous implementation: This is similar to the previous

case except that now an agreement on an issue does not take effect until an agreement
is reached on all the subsequent issues.

Cooperative Models of Multi-Issue Negotiation

1. Simultaneous implementation agenda independence: This axiom states that global

bargaining and sequential bargaining with simultaneous implementation yield the same
agreement.
2. Independent implementation agenda independence: This axiom states that global
bargaining and sequential bargaining with independent implementation yield the same
agreement.

3. Separate/global equivalence: This axiom states that global bargaining and separate
bargaining yield the same agreement.

Non-Cooperative Models of Multi-Issue Negotiation

An agent’s cumulative utility is linear and additive. The functions Ua and Ub give the
cumulative utilities for a and b respectively at time t and are defined as follows.

(4.1)

(4.2)

Where waR+m denotes an m element vector of constants for agent a and wbR+m such a vector
b. These vectors indicate how the agents prefer different issuers. For example, if w ac> wac+1,
then agent a values issue c more than issue c+ 1. Like for agent b.

4.9 ARGUMENTATION

➢ “A verbal and social activity of reason aimed at increasing (or decreasing) the
acceptability of a controversial standpoint for the listener or reader, by putting forward
a constellation of propositions (i.e. arguments) intended to justify (or refute) the
standpoint before a rational judge”

➢ Argumentation can be defined as an activity aimed at convincing of the acceptability of

a standpoint by putting forward propositions justifying or refuting the standpoint.

➢ Argument: Reasons / justifications supporting a conclusion

➢ Represented as: support ->conclusion

– Informational arguments: Beliefs -> Belief e.g. If it is cloudy, it might rain.

– Motivational args: Beliefs, Desires ->Desire e.g. If it is cloudy and you want to get out
then you don’t want to get wet.

– Practical arguments: Belief, Sub-Goals -> Goal e.g. If it is cloudy and you own a
raincoat then put the raincoat.
– Social arguments: Social commitments-> Goal, Desire e.g. I will stop at the corner
because the law say so. e.g I can’t do that, I promise to my mother that won’t.

Process of Argumentation

1. Constructing arguments (in favor of / against a “statement”) from available information.

A: “Tweety is a bird, so it flies”
B: “Tweety is just a cartoon!”
2. Determining the different conflicts among the arguments.
“Since Tweety is a cartoon, it cannot fly!” (B attacks A)
Evaluating the acceptability of the different arguments
“Since we have no reason to believe otherwise, we’ll assume Tweety is a cartoon.”
(accept B). “But then, this means despite being a bird he cannot fly.” (reject A).
3. Concluding, or defining the justified conclusions.
“We conclude that Tweety cannot fly!”
Computational Models of Argumentation:
1. Given the definition of arguments over a content language (and its logic), the models
allow to:
• Compute interactions between arguments: attacks, defeat, support,...
• Valuation of arguments: assign weights to arguments in order to compare them.
Intrinsic value of an argument Interaction-based value of an argument
2. Selection of acceptable argument (conclusion)
• Individual acceptability
• Collective acceptability

4.10 OTHER ARCHITECTURES

LAYERED ARCHITECTURES

Given the requirement that an agent be capable of reactive and pro-active behavior, an
obvious decomposition involves creating separate subsystems to deal with these different types
of behaviors. This idea leads naturally to a class of architectures in which the various
subsystems are arranged into a hierarchy of interacting layers. In this section, we will consider
some general aspects of layered architectures, and then go on to consider two examples of such
architectures:

INTERRAP and TOURINGMACHINES

Typically, there will be at least two layers, to deal with reactive and pro-active behaviors
respectively. In principle, there is no reason why there should not be many more layers.
However many layers there are, a useful typology for such architectures is by the
information and control flows within them. Broadly speaking, we can identify two types of
control flow within layered architectures (see Figure):

• Horizontal layering. In horizontally layered architectures (Figure (a)), the software

layers are each directly connected to the sensory input and action output. In effect, each
layer itself acts like an agent, producing suggestions as to what action to perform.

• Vertical layering. In vertically layered architectures (Figure (b) and (c)), sensory input
and action output are each dealt with by at most one layer each.The great
advantage of horizontally layered architectures is their conceptual simplicity:if we need
an agent to exhibit n different types of behavior, then we implement n different layers.

However, because the layers are each in effect competing with one-another to generate
action suggestions, there is a danger that the overall behavior of the agent will not be coherent.
In order to ensure that horizontally layered architectures are consistent, they generally include
a mediator function, which makes decisions about which layer has ''control" of the agent at any
given time.

The need for such central control is problematic: it means that the designer must
potentially consider all possible interactions between layers. If there are n layers in the
architecture, and each layer is capable of suggesting m possible actions, then this means there
are mn such interactions to be considered. This is clearly difficult from a design point of view
in any but the simplest system. The introduction of a central control system also introduces a
bottleneck into the agent's decision making

Figure 4.12 INTERRAP and TOURINGMACHINES

ABSTRACT ARCHITECTURE

1. We can easily formalize the abstract view of agentgs presented so far. First, we will
assume that the state of the agent’s environment can be characterized as a set S = {s 1,
s2,…) of environment states.
2. At any given instant, the environment is assumed to be in one of these states. The
effectoric capability of an agent is assumed to be represented by a set A = (a1, a2,…)
of actions. Then abstractly, an agent can be viewed as a function

Action S* → A

3. Which maps sequences of enviornment states to actions. We will refer to an agent

modelled by a function of this form as a standard agent. The intuition is that an agent
decides what action to perform on the basis of its history – its experiences to date. These
experiences are represented as a sequence of environment states – those that the agent
has thus for encountered.

4. The (non-determinstic) behaviour of an environment can be modelled as a function env:

S x A → P (S)

Which takes the current state of the environment s  S and an action (perfrmed by the
agent), and maps them to a set of environment state env(s,a) – those that could result
from performing action a in state s. If all the sets in the rnage of env are all sigletons,
(i.e., if the result of performing any action in any state in a set containing a single
member), then the environment is deterministic, and its behaviour can be accrately
predicted.

5. We can represented the interaction of agent and environment as a history. A history h

is a sequence: h:: s 0 a s 1a s 2a s 3a... a  su a...
0 1 2 3 u1 u

where so is the initial state of the environment (i.e., its state when the agent starts
executing), au is the uth action that the agent chose to perform, and s u is the uth
environment state (which is one of the possible results of executing action a u-2 in state
su-1). S* → A is an agent, env: S x S → p(S) is an environment, and so is the initial state
of environment.

6. The characteristic behaviour of an agent action S* → A is an envioronment env : is the

set of all the histories that satisfy these properties. If some property  holds of all these
histories, this property can be regarded as an invariant property of the agent in the
environment. For example, if our agent is a nuclear reactor controller, (i.e., the
environment is a nuclear reactr), and in all possible histories of the contrller/reactor, the
reactor does not below up, then this can be regarded as a (desirable) invariant property.
We will denote by hist(agent, environment) the set of all histories of agent in
environment. Two agentws ag1 and ag2 are said to be behaviorally equivalent with
respect to enviroment env iff hist(ag1, env) = hist(ag2, env) and simply behaviorally
equivalent iffr they are behaviorally equivalent with respect of all environments.

4.11 CONCRETE ARCHITECTURES FOR INTELLIGENT AGENTS

1. We have considered agents only in the abstract. So, while we have examined the properties
of agents that do and do not maintain state, we have not stopped to considerwhat this
state might look like. Similarly, we have modelled an agent's decision makingas an
abstract function action, which somehow manages to indicate which action to
perform—but we have not discussed how this function might be implemented. In this
section, we will rectify this omission. We will consider four classes of agents:

• logic based agents—in which decision making is realized through logical deduction;

• reactive agents—in which decision making is implemented in some form of direct

mapping from situation to action;

• belief-desire-intention agents—in which decision making depends upon the

manipulation of data structures representing the beliefs, desires, and intentions of the
agent; and finally,

• layered architectures—in which decision making is realized via various software

layers, each of which is more-or- less explicitly reasoning about the environment at
different levels of abstraction.

In each of these cases, we are moving away from the abstract view of agents, and
beginning to make quite specific commitments about the internal structure and operation of
agents. Each section explains the nature of these commitments, the assumptions upon which
the architectures depend, and the relative advantages and disadvantages of each.