Robot See, Robot Do : An Overview of Robot Imitation

Paul Bakker and Yasuo Kuniyoshi

Autonomous Systems Section
Intelligent Systems Division
Electrotechnical Laboratory
Tsukuba Science City
Japan
bakker,kuniyosh@etl.go.jp

Abstract interaction with the environment, appropriate

behaviours for itself (e.g., Drescher, 1991).
There are currently two major approaches to Both approaches have proven to be insu-
robot teaching: explicitly tell the robot what to cient to control robot behaviour in practice.
do (programming) or let the robot gure it out Robot programming is simply too hard; it is
for itself (reinforcement learning/genetic algo- dicult and tedious to describe complex be-
rithms). In this paper we give an overview of a haviours to robots in sucient detail, and to
new approach, in which the robot instead learns specify exhaustively how they should adapt to
novel behaviours by observing the behaviour of novel situations.
others: imitation learning. We summarize the
Robot learning, on the other hand, presently
psychological background of this approach, pro-
lacks power. Robots are expected to learn too
pose a de nition of imitation, and identify the
much with too little information. The learn-
important issues involved in implementing imi-
ing paradigms that have so far been employed
tation in robotic systems. Based on this frame-
(such as reinforcement learning and genetic al-
work, we review recent published work in this
gorithms) have proven too weak to learn any
area, and describe an imitation project cur-
behaviours of interest. While they have the
rently underway at the Electrotechnical Labo-
theoretical potential to learn everything, in
ratory in Japan.
practice they are apparently unable to learn
anything (Brooks & Mataric, 1993).
Introduction The recent work in so-called behaviour-based
robotics, and the related animat approach
Ever since the conception of robots, researchers (Maes, 1993) is an attempt to scale back the
have been faced with the problem of how to learning task confronting the robot. It does
make them behave. That is, how to endow this by using a combination of the program-
robots with the ability to perform complex be- ming and learning approaches: a robot is
haviours and interact intelligently with the en- programmed with a set of basic behaviours,
vironment. and is expected to learn how to coordinate
The two most widely-used solutions to this these behaviours so as to maximize perfor-
problem have taken diametrically opposed ap- mance (Brooks & Mataric, 1993). While this
proaches: either tell the robot in detail what it approach has had some good results in learning
has to do (e.g., robot programming), or, at the low-level reactive behaviours, serious questions
other extreme, provide the robot with a simple have been raised about its capacity to scale
learning strategy and let it gure out, through to higher-level intelligent behaviour (Tsotsos,
To Appear in AISB'96 Workshop on Learning in Robots and Animals 2

1995). What Exactly is Imitation?

In this paper, we examine a new approach While imitation can be considered a new topic
to robot learning that similarly takes an inter- in robotics, it has a long history of study and
mediate stance between learning and program- controversy in the elds of ethology and ex-
ming. This approach is learning by imitation. perimental psychology. The controversy has
Put simply, it enables the robot to learn new mainly centred on a question that may prove
behaviours by observing other agents { be they to be of prime importance in robot work as
human or robot { operating in its environment. well: what does `imitation' really mean?
Instead of learning in solitude, agents are now By resolving this contentious issue at an
able to bene t from the experience of others. early stage, we can hopefully avoid the termi-
One could either view imitation as an improved nological problems that have plagued the study
learning paradigm, that provides the learning of imitation in psychology (Galef, 1988). It will
robot with richer information about its envi- also give us a clear foundation from which to
ronment, or alternatively as a simpler approach discuss the promise of imitation in robotics and
to programming that allows one to communi- to evaluate the contributions of recent work.
cate new behaviours to the robot by `showing' At the turn of the century, great interest
instead of `telling'. was generated by the question of whether an-
imals shared any of man's advanced cognitive
A spate of recent papers on robot imita- abilities1 . One of the cognitive abilities in ques-
tion by researchers in Japan, Europe and Aus- tion was learning by imitation.
tralia (Hayes & Demiris, 1994; Kuniyoshi, In- Imitation was succinctly de ned at the time
aba, & Inoue, 1994; Kuniyoshi, 1994; Dauten- to be:
hahn, 1995) re ects the current interest in the
robot community in the potential of this new From an act witnessed learn to do an
approach. act (Thorndike, 1911, p. 79).

As the rst steps are now being taken to- Observation of animal behaviour seemed to
wards implementing imitation in autonomous con rm many clear cases of imitation. For ex-
robots, we considered it timely to prepare an ample, chicks following their mother's example
introductory overview of the concept of im- in avoiding roads, feeding only in certain ar-
itation. Imitation is an established { and eas, and eating certain species of plants; birds
controversial { topic in psychology. The is- learning to puncture and feed from milk bot-
sues highlighted in the psychological literaturetles; and kittens, exposed to adult cats that
are discussed in the next section, where we attained food by manipulating levers, learning
also propose a mechanistic de nition of imita- to perform the same manipulations much faster
tion. Subsequent sections of this paper review than a control group (Galef, 1988).
the potential bene ts of imitation learning for The evidence seemed compelling, but it was
robots, and outline the major issues involved eventually realized that the behaviours de-
in the implementation of imitation learning. A scribed above were not necessarily the result of
suggested framework for mapping progress in `witnessing' an act and learning it: they could
robot imitation is then introduced, and is used be more parsimoniously explained by an inter-
to brie y review the published work on robot play of simpler mechanisms such as reinforce-
imitation to date. ment learning, following behaviour, social facil-
itation, matched dependent behaviour and stim-
Finally, we describe an ongoing project at ulus enhancement.
the Electrotechnical Laboratory in Japan, that 1
The answer to this question was considered at the
aims to have a robot develop the ability to im- time to be an important test of evolution theory { see
itate. Galef (1988).
To Appear in AISB'96 Workshop on Learning in Robots and Animals 3

In the cat experiment cited above, for ex- Imitation takes place when an
ample (Chesler, 1969), the kittens may have agent learns a behaviour from ob-
been more likely to manipulate the lever sim- serving the execution of that be-
ply because the adult cat had left a scent on haviour by a teacher.
it. A chance pawing of the lever then led to
reinforcement and subsequent repetition of the Note that the roles of `teacher' and `agent'
behaviour. Similarly, many of the apparent im- are not xed; they can be reversed from one en-
itatory behaviours of birds can be explained by counter to the next, for example during bouts
the innate following behaviour of chicks. of mutual imitation (Piaget, 1962).
Such alternative learning mechanisms, which
together have the same behavioural result as im- The Promise of Imitation
itation (i.e., the spread of similar behaviours
amongst animals) were unfortunately (and con- In this section we discuss the reasons why we
fusingly) labelled as special cases of imitation should be interested in endowing robots with
in the literature: instinctive imitation, pseudo- the ability to imitate. What speci c advan-
imitation, re ective imitation and so on (Galef, tages would such a learning mechanism give?
1988). Imitation, as de ned above, was rela- As will become clear below, the advantages
belled `true imitation'! are varied and quite substantial, both for in-
It is our submission that in robot imitation dividual agents and for societies of interacting
we are chie y interested in `true imitation', as agents.
described by Thorndike: a behaviour is ob-
served, understood, and reproduced. Our goal Adaptation
is to investigate how robots can be endowed An agent with the ability to imitate has an
with this powerful learning mechanism. Any excellent mechanism for adapting to its en-
other form of `imitation' that does not involve vironment. By observing other agent's ac-
the adoption of a behaviour from the observa- tions, the agent can quickly learn new be-
tion of that behaviour will be precluded from haviours that are likely to be useful; likely,
our discussion.2 because they are already being used by
This is not to say that the simple mecha- agents successfully operating in the same
nisms noted above, which may be used to con- environment.
trol the contagion of behaviours in societies of
robots, are not worthy of study in their own Imitation also acts as an ongoing means of
right. Indeed, we are pursuing this direction adaptation, allowing the agent to induct
in more detail elsewhere (Bakker, 1996b). It new behaviours from fellow agents as the
is simply proposed that they not be included environment changes, new skills are dis-
under the banner of `robot imitation', to avoid covered by other agents, or as the agent
a terminological confusion. moves to a new setting.
Our de nition of imitation is therefore stated Ecient Communication
in terms of the mechanism involved:3
Imitation provides agents with an ecient
2
The term `observation' here includes perception non-verbal means of communication. Be-
cause it is non-verbal, it does not require
through any of the robot's available faculties: \To see or
sense, esp. through directed careful analytic attention"the teacher and the agent to `speak the
(Webster Online Dictionary).
3
Compare this to the more `behaviouristic' de nitionsame language'. This is also true at the
o ered in a psychological text: Imitation is the motoricsomatic level: agents can learn from other
or verbal performance of speci c acts or sounds that agents that are of a di erent species or are
are like those previously performed by a model (Yando, built from di erent hardware.
Seitz, & Zigler, 1978). Such a de nition does not pre-
clude the various types of pseudo-imitation outlined above.
To Appear in AISB'96 Workshop on Learning in Robots and Animals 4

The basic reason for this advantage is that When one agent gains a useful new be-
communication via imitation takes place haviour { be it by trial and error, observ-
at a high level (i.e., in terms of actions) ing a human, or simply from being repro-
rather than at a lower level (such as motor grammed { imitation provides a mecha-
commands). nism for rapidly communicating the dis-
Communication via imitation is also e- covery of this behaviour through the whole
cient because a large amount of impor- society of agents. It provides a means
tant information can be transmitted simul- of combining the power of all the agents'
taneously with each act { the context in diverse learning schemes, to bene t the
which it occurs, the objects that are ma- whole society.
nipulated, the outcomes, and the tools in- Imitation thereby increases the adaptation
volved. and survivability of a society as a whole.
Finally, imitation has the advantage of be- It also ensures the survival of useful be-
ing an undemanding and unobtrusive com- haviours { such behaviours will rapidly
munication method because a teacher does spread, and may even be passed on to fol-
not have to go `o -line' to transfer a be- lowing generations.
haviour to an agent: the agent (or mul-
tiple agents simultaneously) can learn by `Good Company'
observing the teacher without interfering
A nal point is that providing robots with
in the teacher's performance.
imitation ability gives them a skill that
Compatibility with other Learning Mecha- has thus far only been demonstrated in
nisms higher animals { primates, cetaceans and
Imitation can be used as a learning mech- humans. These are also the only animals
anism in conjunction with existing learn- that we consider to exhibit advanced intel-
ing schemes for agents (such as reinforce- ligence. Based on the advantages outlined
ment learning, trial-and-error learning, or above, learning by imitation would seem to
symbolic induction schemes). While the be one of the major components of general
current work examines imitation learning intelligent behaviour. Robots with imita-
in isolation, it can naturally be used as a tory ability may hence nd themselves in
supplemental learning strategy, increasing ethologically `good company' for display-
the agents' learning capabilities overall. ing truly intelligent behaviour.
Ecient Learning
The most signi cant advantage of robot Implementation Issues in
learning by imitation promises to be the Robot Imitation
eciency of the learning process, particu-
larly in a society of agents. This follows After the discussion of the prospective bene ts
from the communication and compatibil- of imitation given above, one might well wonder
ity facets discussed above. what kind of oversight has led to its exclusion
In the traditional learning paradigm for from robot learning thus far! The answer is, of
robots, each agent is `alone' in the envi- course, that there is a good reason why imita-
ronment. All new behaviours must be dis- tion in nature is restricted to higher animals {
covered through personal learning experi- imitation itself requires signi cant perceptual
ence. Imitation opens a rich new vein of and cognitive abilities. Understanding (much
information to the learning robot: the be- less implementing) many of these abilities is
haviours of other agents operating in the still an open problem in psychology, arti cial
same environment. intelligence, and robotics.
To Appear in AISB'96 Workshop on Learning in Robots and Animals 5

Nevertheless, the rst few tentative steps to-  Process the relevant environmental infor-
wards robot imitation have already been taken. mation accompanying the action { the con-
Some of the recent work will be reviewed in the text in which the action occurred, the
next section; the purpose of this section is to participants, the tools or objects manip-
identify the substantive issues that must be ad- ulated.
dressed before robots can be considered able to
imitate. Representation
The following list is based on our own re-
search, and from analyzing the problems ad-  Choose an appropriate representation for
dressed (and purposefully avoided) in recent actions;
work in robotics. While probably not an ex-  Convert an observed action to the agent's
haustive list, it does outline what is at least internal representation (this subsumes an
required to implement imitation in an au- analogy mapping problem { mapping the
tonomous agent. The list also suggests a frame- teacher's actuators to the agent's actua-
work for reviewing future contributions to this tors).
area.
Reproduction
A Conceptual Framework for Robot
 Motivate the agent to consider executing
Imitation
an observed action;
Imitation in robots (or in animals, or humans)
 Choose the appropriate context in which
would appear to be composed of three funda-
mental processes, described by Kuniyoshi et al. to reproduce the action;
(1994) as \seeing, understanding and doing"  Adapt the action to the current environ-
(p. 800). In a reformulation of this statement, ment.
we will propose that for an agent to imitate an
action by a teacher, it must at least:
A Review of Recent Work in
1. observe the action, Robot Imitation
2. represent the action, Here follows a brief review of recent work in
robot imitation. We will discuss these papers in
3. and reproduce the action the context of the framework described above,
highlighting for each paper the issues that were
Each of these fundamental processes in turn addressed, and those which were (purposefully)
involve some important problems: avoided.
Observation Kuniyoshi et al. (1994)

 Motivate the agent to observe a teacher; In Kuniyoshi et al. (1994), a robot agent
watches a human teacher performing a simple
 Identify an appropriate teacher to observe;
assembly task in a tabletop environment. The
 Identify when the teacher is performing an motor movements of the human are classi ed as
action that should be learned; actions known to the robot (pick, move, place
etc.). When the assembly task is completed,
 Accurately observe the teacher's action the robot is commanded to reproduce the se-
(via vision or some other sense { includes quence of actions. It can successfully do this
tracking, attention and segmentation is- even if the initial position of the items to be
sues); manipulated has changed.
To Appear in AISB'96 Workshop on Learning in Robots and Animals 6

Observation. The authors chose to focus stricting the teacher to two essential acts: turn-
on the problem of perceiving the teacher's ac- ing by 90 degrees and moving straight ahead.
tions: determining the start and nish points Representation. The problem of mapping
of actions, and tracking the human hand. This actions from the teacher to the agent is el-
problem is simpli ed by the fact that the robot egantly solved by commanding the agent to
is only required to recognize actions that it al- always follow the teacher. Through the act
ready knows. of following, it must immediately imitate the
The robot was not required to choose which teacher's locomotive action. This agent ac-
teacher or which actions to observe: all actions tion (turning or moving straight ahead) is then
had to be observed in `seeing' mode, and then stored with the environmental information.
reproduced in `doing' mode. Actions are represented as symbolic rules.
Representation. The problem of mapping The antecedent of such a rule is a description of
action sequences to the robot's actuators was the environment in which an action occurred,
solved by using symbolic labels. Once the robot and the RHS is the action to be executed in
correctly interpreted a human action, the label that environment. For example, right-hand
could be mapped to a pre-programmed robot turns become associated with an environment
action sequence. in which there is one wall to the left and one
Reproduction. The robot was explicitly straight ahead.
told when to reproduce the observed actions, Reproduction. When the agent attempts
by being switched to `doing' mode. Kuniyoshi to navigate the maze by itself, it constantly
et al. (1994) focused on the problem of how tries to match the perceived environment to the
to adapt the imitated action to the current en- antecedents of stored production rules. The is-
vironment. The initial state of the table was sue of motivation is thus avoided: in this mode,
analyzed, and the parameters of the action se- the agent is always and only looking to repro-
quence were changed to conform to this state duce observed actions.
(e.g., if the position of objects on the table has In summary, Hayes and Demiris (1994) ad-
changed between the seeing and doing modes). dressed two major issues: how to map a teacher
To sum up, Kuniyoshi et al. (1994) addressed action to an agent action, and how to learn
two fundamental problems: how to accurately when to reproduce an observed action. The
perceive teacher's actions, and how to adapt an solutions implemented by Hayes and Demiris
imitated action to the environment in which it (1994) are simple and elegant, but only work
is reproduced. because of the simplicity of the maze environ-
ment and the restricted set of locomotive ac-
Hayes and Demiris (1994) tions available.
In Hayes and Demiris (1994), a robot agent
is taught the skill of maze traversal by imita- Dautenhahn (1995)
tion. The agent follows a teacher robot through
a maze, detecting signi cant teacher actions In Dautenhahn (1995), agents traverse a `hilly
(such as turning), and physically copying those landscape', attaching themselves to teacher
actions. The agent also learns to associate robots and imitating their trajectories. As in
the environment { the position in the maze Hayes and Demiris (1994), imitation is so far
(in terms of local wall positions) { with the limited to the act of following of other agents.
teacher's actions. Observation. Agents are explicitly pro-
Observation. There is only one teacher to grammed to seek out other agents and attach
attend to, and it must be watched (and fol- themselves. Eventually agents learn to recog-
lowed) at all times in training mode. The iden- nize suitable teachers from positive (or nega-
ti cation of teacher actions is simpli ed by re- tive) learning experiences in the past.
To Appear in AISB'96 Workshop on Learning in Robots and Animals 7

Representation. The problem of interpret-

ing teacher actions is implicitly solved by the motivation to observe

fact that the set of possible actions is very sim- Observation who to observe
ple { movement in a given direction { and the which actions to observe
agent robot can map these actions to its own perceiving teacher actions
body by following the teacher. perceiving the relevant context
Reproduction. Learned movements are as- (environment, tools, objects)
sociated with the local gradient in the hilly
landscape. This would allow the agent to gen- representing actions

eralize behaviours to other, similar areas in the Imitation Representation mapping observed actions to the
hilly landscape. agent’s actuators

In summary, Dautenhahn (1995) proposes

an environment for robots to learn social be- motivation to reproduce actions
haviours in which imitation plays a small role. when to reproduce an action
The implementation of the imitation behaviour Reproduction
is similar to Hayes and Demiris (1994) in Legend
adapting an action to the current
context
that imitation is limited to copying the direc- Kuniyoshi et al. (1994)
tional movements of the teacher, and associat- Hayes & Demiris (1994)
Dautenhahn (1995)
ing these movements with the characteristics of
the environment.
Figure 1: A graphical overview of the issues in-
volved in implementing imitation, and the work
Graphical Summary carried out in the papers reviewed here.
Figure 1 reviews the list of identi ed issues in
robot imitation given above, and shows which This is one of several projects involving im-
of these issues have been (partially) addressed itation that are being pursued by members
in the papers reviewed here. Many more of of the Cognitive Robotics Research Group at
the fundamental problems identi ed need to ETL. One of the long-term goals of this group is
be tackled before a general-purpose imitating to produce a tabletop robot (a stationary robot
robot can be realized. Hopefully, the frame- torso with active vision and a grasping arm)
work can be used to make explicit the par- that is able to manipulate items on a table-
ticular problem areas that future works ad- top, communicate e ectively with humans, and
dress in robot imitation, and to measure overall learn new behaviours through imitation.
progress towards that goal. Another major goal is to introduce imita-
tory behaviours into a society of cooperating
autonomous robots, so that agents can commu-
An Overview of an Imitation nicate skills through imitation (Riekki & Ku-
Project at ETL niyoshi, in press).

In this section, we describe an ongoing project The Development of Imitation

at the Electrotechnical Laboratory (ETL) that
addresses one of the critical issues in robot im- One of the critical problems in imitation is how
itation identi ed in the framework above. The an agent maps an observed action to the motor
project aims to produce a control architecture commands required to imitate it. As an exam-
for robots that allows them to develop the abil- ple, if I wave my hands in front of a robot, how
ity to imitate, by approximately recapitulating can it deduce the actions that it must take to
the developmental acquisition of the imitation produce a similar movement with its actuators?
skill by human infants. And what do we mean by `similar' anyway?
To Appear in AISB'96 Workshop on Learning in Robots and Animals 8

If we take an analytical approach, it would these mechanisms cause progression through

appear that the robot needs to invoke a com- Piaget's stages of learning, see our recent re-
plex mapping function. Not only must it be port (Bakker, 1996a).
able to deduce the correspondence between my The major mechanisms underlying imitation
body and its body, and my arm and its arm, it are as follows (Piaget, 1962):
must also be able to map from its registration
of the action (for example, a visual tracking of  Exploration
the moving arm) to the motor command se-  Circular Reactions
quence required to reproduce a similar action
by its robot arm.  Assimilation
Such a mapping is non-trivial. It subsumes
the inverse kinematics problem, where one  Accommodation
must map from a stationary position of one's Exploration is the initial play period in
arm to the joint settings required to achieve which the child explores the set of possible be-
that position. In this case, it's much harder: haviours. Explorations will be limited by nat-
the robot needs to calculate a sequence of joint ural constraints and tempered by re exes. For
settings for its arm from observing (a sequence example, a child might ail her arms or say
of) positions of the teacher's arm. \ba" and \da". Using this mechanism, the
We plan to implement an imitating robot child builds up a repertoire of actions, and reg-
that achieves the same result using a much sim- istrations of those actions, from observing her-
pler strategy. Our approach is inspired by a self perform them.
psychological account of the cognitive strate- Exploration provides the child with a set of
gies used by human children to learn how to base behaviours from which to draw on to pro-
imitate. duce imitatory behaviours, and a set of regis-
Imitation is not an innate behaviour; it must trations with which to compare and recognize
be learned. Children are at rst unable to behaviours.
imitate at all, and can only imitate novel be- The mechanism of Circular Reactions pro-
haviours (with limited success) after the age of foundly shapes infant behaviour. The child is
12 months. motivated to reproduce any act that it observes
Piaget (1962) studied the development of im- and recognizes; that is, any act that it has
itation skills. His essential thesis was that im- already discovered itself. For example, if the
itation is composed of a set of simple mecha- child hears someone say \ba", and it knows how
nisms. These mechanisms interact with each to produce this sound, then it will repeat the
other, and the environment, to produce a pre- utterance. This is also known as conservative
dictable stagewise progression in the child's imitation { the imitation of known behaviours.
ability to imitate. Circular Reactions provide the child with the
Piaget's ndings are curiously convergent motivation to imitate. They even imitate them-
with the recent `animat' approach to robotics, selves. This leads to the repetitive actions that
that aims to implement complex emergent be- are so typical of infant behaviour (e.g., repeat-
haviour through the interaction of simple mech- ing \ba ba ba", or banging a toy).
anisms. Assimilation is the process whereby the
child maps an observed behaviour to the clos-
Piaget's Imitation Mechanisms
est one she has seen before. If the child has
a small repertoire of behaviours, the observa-
Here, we give a brief overview of the mech- tion might be assimilated to one that seems
anisms hypothesized by Piaget, and describe quite di erent. For example, juggling is repro-
how they interact to produce imitatory be- duced as ball throwing; or \lala" is reproduced
haviour. For a more detailed discussion of how as \dada".
To Appear in AISB'96 Workshop on Learning in Robots and Animals 9

Assimilation provides the child with a crude Importantly, this teleological approach to be-
means of selecting an action to perform as a haviour induction is physically grounded in the
circular reaction. It is based purely on the ob- rst stage of learning, where the agent explores
servation of the outcome of an action, and not the space of possible behaviours allowed by its
on any understanding of the process involved. physical embodiment, and builds up a reper-
The mechanisms of exploration, circular re- toire of behaviours.
actions and assimilation are sucient to allow A computational architecture for controlling
the infant to reach stage 3 of development { the development of imitation has been speci-
where conservative imitation is performed with ed along the lines sketched out above. As a
accuracy. rst experiment, we are applying this control
In stage 4, the step is made to `true' imi- architecture to the learning of speech sounds
tation { the ability to learn novel behaviours by a robot. The agent is provided with hearing
through imitation. The trigger for this is capabilities and an articulatory synthesizer (a
the maturing mechanism of Accommoda- model of the human vocal tract) with which
tion: the ability to detect and correct the dif- to produce speech. It is allowed to explore
ferences between an observed behaviour and its range of vocalizations, and store the re-
one's own reproduction of it. For example, an sulting sounds. In interaction with a human
adult says \barn" and the child responds with teacher, the robot will engage in conserva-
\bah". The development of a more discriminat- tive imitation, eventually tailor its set of ut-
ing representation of speech sounds allows the terances (in the rst instance, monosyllables)
child to perceive the di erence between these to the teacher's language, and by the fourth
two utterances, and the mechanism of accom- stage engage in true imitation of novel speech
modation adjusts her speech act gradually un- sounds. If successful, this will demonstrate for
til it ts the desired outcome. For example, the rst time a learning scheme that allows
the child might try \bah", \bag", \bahg", and robots to learn truly novel behaviours (Brooks
nally \barn". & Mataric, 1993).
In the physical domain, a child might observe
a teacher clicking his ngers. He tries tapping
his ngers together { a known action { and ex-
periments with adjusting his behaviour from Conclusion
there. Success is not guaranteed, and would
depend on the level of skill so far attained, and Robot learning is a eld full of promise, but
the continued help and encouragement of the fraught with intractable problems. Imitation
teacher. learning o ers a new approach that promises a
richer environment for the learning robot, pro-
Discussion
vides a simpler channel of communication be-
tween humans and robots, and may nally em-
By pursuing the developmental route, we hope power robots to induct novel behaviours. The
to solve one of the critical problems of imita- aim of this paper has been to review both the
tion: how one maps a novel observed action to promise and the problems of this burgeoning
one's own body. It is assumed, in our model, new eld in robotics. While recent work is
that this cannot be done for a completely novel encouraging, there are still many deep prob-
action: the observed action must rst be map- lems in perception and representation to be ad-
pable to one that the agent already knows, and dressed before general imitation learning will
then adapted stepwise from there. In other become a reality in robots. Recent work at
words, and conforming to the old adage, it is ETL is drawing on insights from developmental
not possible to learn a new behaviour unless psychology to address some of these problems
one almost knows it already. in a practical way.
To Appear in AISB'96 Workshop on Learning in Robots and Animals 10

Acknowledgements A. Borkowski & J. L. Crowley (Eds.),

Proceedings of the 2nd International
This work was generously supported by the Sci- Symposium on Intelligent Robotic Sys-
ence and Technology Agency program of the tems, pp. 198{204. Grenoble, France:
Japanese government. Our ideas have bene- LIFTA-IMAG.
ted greatly from discussions with visitors and
colleagues at ETL: Alex Zelinsky, Polly Pook, Kuniyoshi, Y. (1994). The science of im-
Luc Berthouze, and Jukka Riekki. Thanks also itation { towards physically and so-
to John Demiris and Kerstin Dautenhahn for cially grounded intelligence. Tech. Rep.
ongoing email contact and encouragement. RWC TR-94001, Real World Comput-
ing Project, Tsukuba-shi, Ibaraki-ken,
Japan.
References
Kuniyoshi, Y., Inaba, M., & Inoue, H.
Bakker, P. (1996a). A description of a robotic (1994). Learning by watching: Extract-
progression through the rst four devel- ing reusable task knowledge from visual
opmental stages of imitation: Cognitive observation of human performance. IEEE
mechanisms. Manuscript in preparation. Transactions on Robotics and Automa-
Bakker, P. (1996b). Simple mechanisms for be- tion, 10 (6), 799{822.
haviour contagion in autonomous robots. Maes, P. (1993). Behavior-based arti cial in-
Manuscript in preparation. telligence. In J. A. Meyer, H. L. Roit-
Brooks, R. A., & Mataric, M. J. (1993). blat, & S. W. Wilson (Eds.), From Ani-
Real robots, real learning problems. In mals to Animats 2, pp. 2{10. Cambridge,
J. H. Connell & S. Mahadevan (Eds.), MA: MIT Press.
Robot Learning, pp. 193{213. Boston, Piaget, J. (1962). Play, Dreams and Imitation
MA: Kluwer Academic. in Childhood. New York: W. W. Norton.
Chesler, P. (1969). Maternal in uence in learn- (Original work published 1945).
ing by observation in kittens. Science, Riekki, J., & Kuniyoshi, Y. (in press). Be-
166, 901{903. havior cooperation based on markers and
Dautenhahn, K. (1995). Getting to know each weighted signals. To appear in Interna-
other { arti cial social intelligence for tional Symposium on Robotics and Manu-
autonomous robots. Robotics and Au- facturing , Montpelier, France, May 1996.
tonomous Systems, 16, 333{356. Thorndike, E. L. (1911). Animal Intelligence.
Drescher, G. L. (1991). Made-Up Minds: A New York: Macmillan. Cited in Galef
Constructivist Approach to Arti cial In- (1988).
telligence. Cambridge, MA: MIT Press.
Tsotsos, J. K. (1995). Behaviorist intelligence
Galef, B. G. (1988). Imitation in animals: and the scaling problem. Arti cial Intel-
History, de nition and interpretation of ligence, 75 (2), 135{160.
data from the psychological laboratory.
In T. R. Zentall & B. G. Galef (Eds.), Yando, R., Seitz, V., & Zigler, E. (1978). Imita-
Social Learning: Psychological and Bio- tion: A Developmental Perspective. Hills-
logical Perspectives, pp. 3{28. Hillsdale, dale, NJ: Lawrence Erlbaum.
NJ: Lawrence Erlbaum.
Hayes, G., & Demiris, J. (1994). A robot con-
troller using learning by imitation. In