National Institute of Technology Jamshedpur

NATIONAL INSTITUTE OF TECHNOLOGY
JAMSHEDPUR
A
RESEARCH PAPER
ON
LEARNING OF ROBOTS BY USING
AND SHARING THEIR EXPERIENCES
SUBMITTED TO:Mrs. LUCY SINGH

TECHNICAL COMMUNICATION
ABSTRACT
SUBMITTED BY:Abhishikt V V
MTECH (ISSE)
2015PGCAIS04
Robots are shaping a new era in the technology, many algorithms has been developed for their
learning which they can apply to find the solution for the problems that they are facing. That is, we
have tried to provide them a basic intelligence (like recognition, collision avoidance, etc.). But besides
that basic intelligence we are interested in developing an intelligent system in which robots will use
their experiences and share it with each other just like humans, who have got all its intelligence by his
experience and imaginations and sharing it with each other. To implement this intelligent system we
can use a local server (word interchangeable to database) that will be restricted to a robot only and a
global server where the authorized robots can upload their experiences which can be used by every
robot (which is authorized to that server) to solve their problems
1. INTRODUCTION
Washburne (1936) defined learning as a process of acquiring knowledge, technique, attitude and value
through instruction and experience. Robot Learning is a technique that enables a robot to adjust itself
in the environment. It also enables the robot to deal with the unexpected situations that it had never
experienced in its past. In earlier decades the learning was based on teaching to a single robot with the
algorithms based learning. But recent developments in the technology (i.e. increase in computer &
internet speed) made learning of a robot from its experience a promising research topic. An additional
advantage of learning a robot from its experience is that it can also learn from the experience of its
peers. A key feature is that, we are using two servers one is the local server restricted to a robot only
while other is the global server which is shared globally with all the robots. And both these servers
will hold the experiences of the robots.
2. RELATED WORK & PREVIOUS WORK

2.1 RoboEarth
The RoboEarth project (i.e. World Wide Web for robots) is implementing the global server for sharing
the experiences of robot. The objective of RoboEarth Project is to enable robots from all over the
world to share their knowledge about actions, objects and the environment the RoboEarth platform
offers the ability to store reusable data in a generic and open format.
2.2 Optimization Learning Algorithms

Many algorithms have been developed for optimizing the behaviour of a machine. But here we are
using reinforcement learning algorithm for optimizing performances of robots.
2.2.1
Reinforcement Learning
Reinforcement learning is the study of how animals and the machines can learn to optimize their
behaviour in the face of rewards and punishments. Reinforcement learning algorithms have been
developed that are closely related to methods of dynamic programming, which is a general approach
to optimal control. Reinforcement learning phenomena have been observed in psychological studies
of animal behaviour, and in neurobiological investigations of neurons-modulation and addiction.
Reinforcement learning algorithm consist of
Set of Environment States E

Set of actions required A (Solution of a problem may consists of sequence of actions)
Rules of transition between states
Rules that determine reward for the transition.
Rules that describe what the agent observes.
3. PROPOSED APPROACH
To realize this whole scenario we will use two servers one is local to a particular robot, while
other is centralized server which can be accessed by every robot which is authorized to that
server. The purpose of making two servers is that the local server will handle or assist the robot to
work in a particular environment and with the particular objects and as soon as the robot will
continue to work in the same environment it will gain experience of working in that environment
and hence it will work more intelligently while the global server will keep the solution for all the
problems (tasks assigned to robots are referred as problems) which are faced by all the robots
present globally, and these solutions will be generalized rather than specialized e.g. a robot
working in your garden and trimming a plant, if we look at the optimal solution then your robot
will be trimming the plant according to the size and shape of that particular plant. But the robot at
the remote location working in another garden will have slightly different environment
(difference may be very slight) and different specification of object of interaction (i.e. in this
case, shape & size of plant). So the solution of first robot cannot be used by second robot directly.
Hence the solution of first robot will be considered as generalized one at the global server and the
robot at remote location will use this solution after applying few optimization algorithms and
making slight changes in the environment variables, and specifications of object of interaction etc
as per its requirement.
3.1 Local Server (Local Database)

The scope of creating a local database is that every robot is having a different environment to
work and they deal with different-different objects. As the robot will deal with the objects in its
surroundings, it will gain experience of dealing with these objects by optimizing the solution on
the basis of its environment, object of interactions and saving this solution in its local database.
So the local database will act as memory of a robot to let it work more intelligently in a particular
environment e.g. a robot working in a hospital will have experiences about dealing with the
objects present in that hospital, these experiences will get collected in the local database and as
the time passes the robot will have good intelligence to handle every object (of interaction) in the
hospital expertly.
3.1.1 Problem Characteristics & Variables for Local Database (i.e. characteristics of the
task assigned to a robot)
Problem Id (PID) - Every problem will have identification number.

Problem Statement We can use it as the key for searching the problem present in the
database.
Environment Description about the working condition (i.e. hospital, railway platform etc)
Environment Id attached to a problem (EID) ID generated for different-different
environments of the same problem.
Description of Object of interaction Object, to which the robot is interacting may be
Door, table, train etc
3.1.2 Algorithm for the Local Server
Step1 Check whether the problem statement already exists or not. If no go to step 6.
Step2 - Check for the environment condition matches or not. If no, go to step 7.
Step3 Check for the solution is optimal or not. If yes, then go to step4 else go to step5.
Step4 Use the solution and Increase the Memory function by 1 and Exit.
Step5 Increment the Memory function by 1, Apply reinforcement learning algorithm to find the
more optimal solution, use it and Exit.
Step6 Create the problem Statement & problem Id (PID).
Step7 Create an environment Id (EID) corresponding to the PID of the problem.
Step8 Initialize the environment variable according to the environment.
Step9 Describe the object of interaction. (Table, fan etc)
Step10 Initialize the memory function by 1 and trigger downloading (searching) from the global
server.
Step11 If the global server returns some solution then apply the reinforcement learning algorithm to
make it optimal for its own use. Save it in its local database and exit.
Step12 If the global server returns NULL then the robot will apply its own learning algorithm and
find the solution. If it gets success to find the solution (i.e. task completed) then trigger uploading this
solution at the global server as well as at the local server.
Step13 Exit
3.1.3 Constraints/Protocols
If the solution is optimal then we will not call the optimization learning algorithm for
optimizing it again we will consider that solution as Final solution.
Memory Function corresponding to a particular problem will define how many times the
robot has performed that task.
3.2 Problem Characteristics & variables for global database
Global problem Id (GPID) Identification of the problem globally corresponding to the

problem statement.
Problem Statement - Key for searching the problem.
Environment Description about the working condition (i.e. hospital, railway platform,
library etc)
Global Environment Id attached to a problem (GEID) Generated for different-different
environments of the same problem.
Description of Object of interaction to which the robot is interacting, may be Door, rain,
train etc or it can be null.
3.2.3 Algorithms for global server

The global server has two main tasks one is uploading the solution received from the robot to its
database and second one is granting the permission for downloading request made by robot. The
algorithms are as follows3.2.3.1 Algorithm for Uploading
Step1 Create a GPID for the problem and GEID for the environment.
Step2 Get the values of various variables like problem statement and the environment variables,
object of interaction etc from the robot.
Step3 The global server will generalize this solution by applying some algorithms.
Step4 Save it at global server.
Step5 Exit.
3.2.3.2 Algorithm for Downloading

Step1 Check for the problem and environment variables at the global server if the solution
corresponding to this problem environment set exists there then download the solution.
Step2 Specialize this solution by making slight changes in its variables as required and apply the
reinforcement learning algorithm for optimization according to its own environment.
Step3 Use the solution, initialize memory function and save it at local database.
Step4 Exit.
3.2.3 Constraints/Protocols
Fuzzy logic and various matching algorithms may be used for the match when the robot is
looking for the solution at the global server.
Only authorized Robots can access the global server.
4. CONCLUSION & FUTURE WORK

We have seen that how we can make robots more intelligent to work in a particular environment by
enabling them to use their own experience to deal with the objects present in their surrounding
environments i.e. as soon as the time passes the robot will have a good experience of dealing with the
objects present in their surroundings. We have also seen that how a robot will use the experience of
other robots to solve its own problem by using the global server. In near future we can also work on
developing an intelligent agent for Robot Human interaction by having the similar approach that we
have seen in this paper.
5. REFERENCES
[1] Chun-Wang WEI, I-Chun HUNG, Ling LEE & Nian-Shing CHEN April 2011 A joyful classroom
learning system with robot learning companion for children to learn mathematics multiplication.
[2] ICT Call 4 RoboEarth Project 2010-248942 Deliverable D6.1: Complete specification of the
RoboEarth platform December 1, 2010.
[3] Peter Dyan Gatsby, University College London & Christopher JCH Watkins, University of
London: Reinforcement Learning.
[4] Wikipedia: Reinforcement Learning, 20th November 2011
http://en.wikipedia.org/wiki/Reinforcement_learning.

National Institute of Technology Jamshedpur

Uploaded by

Copyright:

Available Formats

National Institute of Technology Jamshedpur

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

National Institute of Technology Jamshedpur

Uploaded by

Copyright:

Available Formats

NATIONAL INSTITUTE OF TECHNOLOGY

SUBMITTED TO:Mrs. LUCY SINGH

2. RELATED WORK & PREVIOUS WORK

2.2 Optimization Learning Algorithms

Set of Environment States E

3.1 Local Server (Local Database)

Problem Id (PID) - Every problem will have identification number.

3.2 Problem Characteristics & variables for global database

Global problem Id (GPID) Identification of the problem globally corresponding to the

3.2.3 Algorithms for global server

3.2.3.2 Algorithm for Downloading

Only authorized Robots can access the global server.

4. CONCLUSION & FUTURE WORK

You might also like