Intelligent Agents

Chapter 2
 Outline
• Agents and environments
• Rationality
• PEAS (Performance measure,
Environment, Actuators, Sensors)
• Environment types
• Agent types
 Agents
• An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment
through actuators

• Human agent: eyes, ears, and other organs for sensors; hands,legs, mouth, and other
body parts for actuators
• various motors for actuators
• Robotic agent: cameras and infrared range finders for
sensors;
 2.1 AGENTS AND ENVIRONMENTS
• The term percept refers to the agent’s perceptual
inputs at any given instant.
• An agent’s percept sequence is the complete
history of everything the agent has ever perceived.
• An agent’s behavior is described by the agent
function that maps any given percept sequence to
an action.
• The agent program is a concrete implementation,
running within some physical system.
 Agents and environments

• The agent function maps from percept histories

to actions:
[f: P*  A]
• The agent program runs on the physical
architecture to produce f
• agent = architecture + program
•
Example of vacuum-cleaner world
Agents and environments
 A vacuum-cleaner agent
• \input{tables/vacuum-agent-function-table}
•
 Rational agents
• An agent should strive to "do the right thing",
based on what it can perceive and the actions it
can perform. The right action is the one that will
cause the agent to be most successful
• Performance measure: An objective criterion for
success of an agent's behavior
• E.g., performance measure of a vacuum-cleaner
agent could be amount of dirt cleaned up,
amount of time taken, amount of electricity
consumed, amount of noise generated, etc.
•
•
 Rational agents
• Rational Agent: For each possible percept
sequence, a rational agent should select
an action that is expected to maximize its
performance measure, given the evidence
provided by the percept sequence and
whatever built-in knowledge the agent has.
•
 Rational agents
• Rationality is distinct from omniscience (all-
knowing with infinite knowledge)
• Agents can perform actions in order to
modify future percepts so as to obtain
useful information (information gathering,
exploration)
• An agent is autonomous if its behavior is
determined by its own experience (with
ability to learn and adapt)
•
 PEAS
• PEAS: Performance measure, Environment,
Actuators, Sensors
• Must first specify the setting for intelligent
agent design
– Performance measure
– Actuators
– Sensors
– Environment
• Consider, e.g., the task of designing an
automated taxi driver:
 PEAS
• Must first specify the setting for intelligent agent
design
– Performance measure: Safe, fast, legal, comfortable
trip, maximize profits
– Environment: Roads, other traffic, pedestrians,
customers
– Actuators: Steering wheel, accelerator, brake, signal,
horn
– Sensors: Cameras, sonar, speedometer, GPS,
odometer, engine sensors, keyboard
• Consider, e.g., the task of designing an
automated taxi driver:
 PEAS
• Agent: Medical diagnosis system
– Performance measure: Healthy patient,
minimize costs, lawsuits
– Environment: Patient, hospital, staff
– Actuators: Screen display (questions, tests,
diagnoses, treatments, referrals)
– Sensors: Keyboard (entry of symptoms,
findings, patient's answers)
 PEAS
• Agent: Part-picking robot
– Performance measure: Percentage of parts in
correct bins
– Environment: Conveyor belt with parts, bins
– Actuators: Jointed arm and hand
– Sensors: Camera, joint angle sensors
 PEAS
• Agent: Interactive English tutor
– Performance measure: Maximize student's
score on test
– Environment: Set of students
– Actuators: Screen display (exercises,
suggestions, corrections)
– Sensors: Keyboard
 Environment types
• Fully observable (vs. partially observable): An agent's sensors give it
access to the complete state of the environment at each point in time.
• Deterministic (vs. stochastic): The next state of the
environment is completely determined by the current
state and the action executed by the agent. (If the
environment is deterministic except for the actions of
other agents, then the environment is strategic)
• Episodic (vs. sequential): The agent's experience is
divided into atomic "episodes" (each episode consists of
the agent perceiving and then performing a single
action), and the choice of action in each episode
depends only on the episode itself.
•
 Environment types
• Static (vs. dynamic): The environment is
unchanged while an agent is deliberating. (The
environment is semidynamic if the environment
itself does not change with the passage of time
but the agent's performance score does)
• Discrete (vs. continuous): A limited number of
distinct, clearly defined percepts and actions.
• Single agent (vs. multiagent): An agent
operating by itself in an environment.
•
•
 Environment types
Chess with Chess without Taxi
driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

• The environment type largely determines the agent design

• The real world is (of course) partially observable, stochastic,
sequential, dynamic, continuous, multi-agent
•
•
 Agent functions and programs
• An agent is completely specified by the
agent function mapping percept
sequences to actions
• One agent function (or a small
equivalence class) is rational
• Aim: find a way to implement the rational
agent function concisely
•
•
 Table-lookup agent
• \input{algorithms/table-agent-algorithm}
• Drawbacks:
– Huge table
– Take a long time to build the table
– No autonomy
– Even with learning, need a long time to learn
the table entries
•
 Agent program for a vacuum-
cleaner agent
• \input{algorithms/reflex-vacuum-agent-
algorithm}
•
 Agent types
• Four basic types in order of increasing
generality:
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
•
 Simple reflex agents

• These agents select actions on the basis

of the current percept, ignoring the rest of
the percept history. For example, the
vacuum agent, whose agent function is a
simple reflex agent, because its decision is
based only on the current location and on
whether that location contains dirt.
Simple reflex agents
 Simple reflex agents
• \input{algorithms/d-agent-algorithm}
•
 Model-based reflex agents
In this, the agent should maintain some sort of internal state
that depends on the percept history and thereby reflects at
least some of the unobserved aspects of the current state.
• State: For the braking problem, the internal state is not too
extensive— just the previous frame from the camera,
allowing the agent to detect when two red lights at the edge
of the vehicle go on or off simultaneously.
•How the world evolves?: For example, an overtaking car
generally will be closer behind than it was a moment ago.
•What my actions do?: ARer driving for five minutes towards
Makkah on the Highway, one is usually about five miles
towards Makkah of where one was five minutes ago.
Model-based reflex agents
 Model-based reflex agents
• \input{algorithms/d+-agent-algorithm}
•
 Goal-based agents
• The agent needs some sort of goal
information that describes situations that
are desirable—For example, at a road
juncIon, the taxi can turn leR, turn right, or
go straight on. The correct decision
depends on where the taxi is trying to get
to. – The agent program can combine this
with the model- based reflex agent to
choose acIons that achieve the goal.
 Goal-based agents

•
 Utility-based agents –
• UtiIlity-based agents addresses the partial
observability and stochasticity and
decision making under uncertainty
Utility-based agents
– Learning allows the agent to operate in initially unknown environments.
– The performance element is responsible for selecting external actions.
– The learning element is responsible for making improvements. It takes
feedback from the critic on how the agent is doing and determines
changes for the performance element to do better in the future.
– Critic tells the learning element how well the agent is doing with respect
to a fixed performance standard.
– If the agent is willing to explore a little and do some suboptimal actions
in the short run, to help much better actions for the long run. The problem
generator’s job is to suggest these exploratory actions.
Learning agents