THE NEURO-ROBOTICS PARADIGN

1. INTRODUCTION
Neurorobotics, a combined study of neuroscience, robotics, and artificial intelligence (AI), is the science and technology of embodied autonomous neural systems. Neural systems include brain-inspired algorithms (e.g. connectionist networks), computational models of biological neural networks (e.g. artificial spiking neural networks, large-scale simulations of neural microcircuits) and actual biological systems (e.g. in vivo and in vitro neural nets). Such neural systems can be embodied in machines with mechanic or any other forms of physical actuation. This includes robots, prosthetic or wearable systems but at also, at smaller scale, micro-machines and, at the larger scales, furniture and infrastructures. Neurorobotics is that branch of neuroscience with robotics, which deals with the study and application of science and technology of embodied autonomous neural systems like braininspired algorithms. At its core, neurorobotics is based on the idea that the brain is embodied and the body is embedded in the environment. Therefore, most neurorobots are required to function in the real world, as opposed to a simulated environment. Neurorobotics represents the two-front approach to the study of intelligence. Neuroscience attempts to discern what intelligence consists of and how it works by investigating intelligent biological systems, while the study of artificial intelligence attempts to recreate intelligence through non-biological, or atificial means. Neurorobotics is the boundary between the two, where biologically inspired theories are tested in a grounded environment, with a physical implementation of said model. The successes and failures of a neurorobot and the model it is built from can provide evidence to refute or support that theory, and give insight for future study.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 1

THE NEURO-ROBOTICS PARADIGN

2. Major classes of neurorobotic models

Neurorobots can be divided into various major classes based on the robot's purpose. Each class is designed to implement a specific mechanism of interest for study. The three common types of neurorobots are those used to study motor control, memory, and action selection.

A. Locomotion and motor control

Neurorobots are often used to study motor feedback and control systems, and have proved their merit in developing controllers for robots. Locomotion is modeled by a number of neurologically inspired theories on the action of motor systems. Locomotion control has been mimicked using models or central pattern generators, clumps of neurons capable of driving repetitive behavior, to make four-legged walking robots. Other groups have expanded the idea of combining rudimentary control systems into a hierarchical set of simple autonomous systems. These systems can formulate complex movements from a combination of these rudimentary subsets. This theory of motor action is based on the organization of cortical columns, which progressively integrate from simple sensory input into complex afferent signals, or from complex motor programs to simple controls for each muscle fiber in efferent signals, forming a similar hierarchical structure. Another method for motor control uses learned error correction and predictive controls to form a sort of simulated muscle memory. In this model, awkward, random, and error-prone movements are corrected for using error feedback to produce smooth and accurate movements over time. The controller learns to create the correct control signal by predicting the error. Using these ideas, robots have been designed which can learn to produce adaptive arm movements or to avoid obstacles in a course.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 2

THE NEURO-ROBOTICS PARADIGN

B. Learning and memory systems

Robots designed to test theories of animal memory systems. Many studies currently examine the memory system of rats, particularly the rat hippocampus, dealing with place cells, which fire for a specific location that has been learned. Systems modeled after the rat hippocampus are generally able to learn mental maps of the environment, including recognizing landmarks and associating behaviors with them, allowing them to predict the upcoming obstacles and landmarks. Another study has produced a robot based on the proposed learning paradigm of barn owls for orientation and localization based on primarily auditory, but also visual stimuli. The hypothesized method involves synaptic plasticity and neuro modulation, a mostly chemical effect in which reward neurotransmitters such as dopamine or serotonin affect the firing sensitivity of a neuron to be sharper. The robot used in the study adequately matched the behavior of barn owls. Furthermore, the close interaction between motor output and auditory feedback proved to be vital in the learning process, supporting active sensing theories that are involved in many of the learning models. Neurorobots in these studies are presented with simple mazes or patterns to learn. Some of the problems presented to the neurorobot include recognition of symbols, colors, or other patterns and execute simple actions based on the pattern. In the case of the barn owl simulation, the robot had to determine its location and direction to navigate in its environment.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 3

THE NEURO-ROBOTICS PARADIGN

C. Action selection and value systems

Action selection studies deal with negative or positive weighting to an action and its outcome. Neurorobots can and have been used to study *simple* ethical interactions, such as the classical thought experiment where there are more people than a life raft can hold, and someone must leave the boat to save the rest. However, more neurorobots used in the study of action selection contend with much simpler persuasions such as self preservation or perpetuation of the population of robots in the study. These neurorobots are modeled after the neuromodulation of synapses to encourage circuits with positive results. In biological systems, neurotransmitters such as dopamine or acetylcholine positively reinforce neural signals that are beneficial. One study of such interaction involved the robot Darwin VII, which used visual, auditory, and a simulated taste input to "eat" conductive metal blocks. The arbitrarily chosen good blocks had a striped pattern on them while the bad blocks had a circular shape on them. The taste sense was simulated by conductivity of the blocks. The robot had positive and negative feedbacks to the taste based on its level of conductivity. The researchers observed the robot to see how it learned its action selection behaviors based on the inputs it had. Other studies have used herds of small robots which feed on batteries strewn about the room, and communicate its findings to other robots.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 4

THE NEURO-ROBOTICS PARADIGN

Biological robots
These are not officially a neurorobot in that they are not neurologically inspired AI systems, but actual neuron tissue wired to a robot. This employs the use of cultured neural networks to study brain development or neural interactions. These typically consist of a neural culture raised on a multi electrode array (MEA), which is capable of both recording the neural activity and stimulating the tissue. In some cases, the MEA is connected to a computer which presents a simulated environment to the brain tissue and translates brain activity into actions in the simulation, as well as providing sensory feedback. The ability to record neural activity gives researchers a window into a brain, albeit simple, which they can use to learn about a number of the same issues neurorobots are used for. An area of concern with the biological robots is ethics. Many questions are raised about how to treat such experiments. Seemingly the most important question is that of consciousness and whether or not the rat brain experiences it. This discussion boils down to the many theories of what consciousness is.

Implications for neuroscience

Neuroscientists benefit from neurorobotics because it provides a blank slate to test various possible methods of brain function in a controlled and testable environment. Furthermore, while the robots are more simplified versions of the systems they emulate, they are more specific, allowing more direct testing of the issue at hand. They also have the benefit of being accessible at all times, while it is much more difficult to monitor even large portions of a brain while the animal is active, let alone individual neurons. With subject of neuroscience growing as it has, numerous neural treatments have emerged, from pharmaceuticals to neural rehabilitation.] Progress is dependent on an intricate understanding of the brain and how exactly it functions. It is very difficult to study the brain, especially in humans due to the danger associated with cranial surgeries. Therefore, the use of technology to fill the void of testable subjects is vital. Neurorobots accomplish exactly this,
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011 Page 5

THE NEURO-ROBOTICS PARADIGN

improving the range of tests and experiments that can be performed in the study of neural processes. The neuro-robotics paradigm is a novel design approach, mainly aimed at the fusion of neuroscience and robotic competences and methods to design better robots that can be used in rehabilitation and functional support. The ultimate objective of the neuro-robotics paradigm is very ambitious: to introduce a discontinuity in the robot design, thus going literally beyond robotics. In the last decades neuroscientists obtained important results in using different robotic platforms as a tool for their investigations. More precisely, the robotic device has been used as a reliable and accurate instrument to develop behavioural experiments as illustrated in the following: to measure and record specific parameters of neuroscientific interest (e.g. the position and velocity of the human hand, the impendence of the human arm during quick movement tasks ); to interact with a subject to analyze his/her responses to a specific external stimulus (e.g. the response to a tactile stimulus on the finger tip , the effect of a given force disturbance on the hand trajectory ). In any case, the object of the neuroscientific investigation is the behaviour of the human subject, while the robotic device acts just as a support to measure the relevant parameters. to measure and record specific parameters of neuroscientific interest (e.g.

the position and velocity of the human hand, the impendence of the human arm during quick movement tasks; to interact with a subject to analyze his/her responses to a specific external

stimulus (e.g. the response to a tactile stimulus on the finger tip, the effect of a given force disturbance on the hand trajectory In any case, the object of the neuroscientific investigation is the behaviour of the human subject, while the robotic device acts just as a support to measure the relevant parameters. This is only the simplest way in which neuroscience and robotics can

interact. Indeed two more possible a approaches of neuro-robotics are: development of robotic models (physical platforms) for they investigation of neuroscience theories; in this case, the robot is the object itself of the neuroscientific investigation;

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 6

THE NEURO-ROBOTICS PARADIGN

Neuroscientific theories can be applied to design and develop novel

robotic systems. The bio-inspiration of such devices will push forward the usual morphological bio-inspiration by implementing human motion-control strategies to control the device. Our research laboratory recently focused on the development of a number of robotic systems following the innovative approaches of neuro-robotics. In particular this paper presents three robots: the NEURARM, a robotic model of the human upper limb designed to study neuroscientific motion control theories; the NEUROExos and HANDEXOS, the elbow and hand modules of an upper limb rehabilitation exoskeleton, designed in collaboration with neuroscientists to allow physiological motion of the users elbow and hand during functional support and rehabilitation tasks.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 7

THE NEURO-ROBOTICS PARADIGN

3. ROBOT AS A MODEL: NEURARM

Neuroscience investigations are still ongoing and conducted by a variety of means in human and animal subjects. Among others, several methods based on the recording of mechanical and neural data are commonly used. However, there is an increasing need to test different neuroscientific hypotheses by implementing them on a model system that is under full control of the experimenter. This way, results obtained by 'standard' neuroscience methods can be compared with those obtained from the model system. While this can be achieved to some extent through numerical simulation, these results are only as good as the accuracy of the numerical model conceived by the investigators. As a supporting tool to these mathematical analyses, the implementation of a specific hypothesis on a real mechanical system can reveal the effects of unmodeled dynamics and provide critical insight into how the human system works in a real environment. In order to address the need of a real mechanical model, and to support the investigation of neuroscientific hypotheses, the functionally bio-inspired NEURARM platform was developed (see Figure 3.1).

Figure 3.1: The neuro-robotics design paradigm: from the biological model a simplified cybernetic model, of both the mechanics and the neural control is obtained. This is the starting point for the development a bio-inspired robotic artifact, which can be used by neuroscientists as a simplified model of the human arm DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011 Page 8

THE NEURO-ROBOTICS PARADIGN

The NEURARM system is a 2 link2 degrees of freedom (DoF) planar robotic arm that replicates the main functional parameters of the human upper limb, i.e. its tendon driven agonistantagonist actuation, mass, inertia, and dynamic performance. Clearly, this planar system is a gross simplification of the complexity of the human arm. However, the system is complex enough to address essential questions about human behavior. The two-joint linkage provides significant non-linear kinematics, statics and dynamics while the actuator system provides both redundancies in terms of force and torque production. Indeed, considerable coming into human motor behaviour has been gathered from focused experiments by restricting human movements to the horizontal plane. The functionally bio-inspired actuation system permits to implement motion control algorithms which resemble that of the human arm. A pair of muscles powering the human joint in antagonistic configuration provides the peculiar characteristics of the equilibrium point hypothesis (EPH) for human motor control. Since muscles have a natural stiffness and viscosity that varies with the muscle activation level, the central nervous system (CNS) can generate stable equilibrium postures, towards which the arm is attracted, by properly regulating the activation levels of antagonistic muscles. Moreover, the CNS can generate stable posture and even movements in absence of sensory feedback, by shifting the equilibrium point. By coactivating antagonistic muscle in parallel, the mechanical impedance (i.e. stiffness) can also be regulated. The resulting system is intrinsically stable and robust with respect to the neural transmission delays and has the capability to control on demand the impedance at the hand. The NEURARM platform emulates the antagonist tendon driven actuation system of the human arm. Moreover, it replicates the non linear force elongation characteristic of the muscletendon complex, by means of a contractile element (hydraulic piston) in series with a non-linear elastic element. By properly adjusting the piston positions the control system can specify in an open-loop fashion both the joint equilibrium position and mechanical stiffness. Using this device we can test neuroscientific hypotheses about how the CNS controls movement in free space and the interaction with environment.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 9

THE NEURO-ROBOTICS PARADIGN

4. NEUROSCIENCE IN ROBOTICS: NEUROEXOS AND HANDEXOS

A. NEUROExos

Figure 4.1: Overview of the NEUROExos.

Other than traditional rehabilitation techniques, in the last decades a great deal of effort and attention were concentrated to develop several robotic platforms aimed at the post stroke rehabilitation or as assistive devices of the upper limb of disabled people. While some robotics research groups focused the attention on operative machines rehabilitating the user upper limb by guiding solely his/her hand motion, such as the MIT Manus [1], the MIME [9], the MEMOS [10], the ARM-guide [11], other research groups focused the attention on wearable devices, acting like an upper limb exoskeleton. State of the art is represented by several examples of upper limb exoskeletons. Perry and Rosen designed and developed a multipurpose 7 degrees of freedom (dof) upper-limb exoskeleton. Carignan and colleagues designed and developed a 5-dof
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011 Page 10

THE NEURO-ROBOTICS PARADIGN

arm exoskeleton with passive adjustable linkages. Kiguchi and colleagues [16] built a 3-dof exoskeleton for physically assisting disabled, injured and/or elderly persons. Kousidou, Caldwell and colleagues used the Salford Rehabilitation Exoskeleton (7-dof), for investigating physiotherapy in three dimensional space. Nef and colleagues presented ARMin [18], a 7-dof robot for the rehabilitation of the upper limb. Critical analysis of the state of the art and the close interaction with neuroscientists showed that the crucial aspects in designing an exoskeleton are the localization and distribution of the physical interaction point between the user and the robot and their kinematic coupling. The absence of a perfect kinematic compatibility has two critical drawbacks: the subjects arm joints can be overloaded and it is impossible to provide any assistive strategy or rehabilitative treatment aimed at supporting singularly each user arm joint. The second key issue is that of the actuation system. A rigid transmission permits to easily control the robot in terms of joint or end-effector positions as well as to provide an appropriate torque/force field. However, in order to obtain a backdrivabile system, ensuring a safe interaction between the exoskeleton and the user, a muscle like actuation system could be implemented. Thanks to the muscles-tendon viscoelastic properties human joints, are intrinsically stable and have an actively adjustable passive compliance. These properties are desirable even in robots interacting with humans, especially when operating with disabled people and spastic events could happen. The NEUROExos elbow module was developed with the following three main requirements: Each link is composed by a double shell structure with an inner layer

perfectly fitting the user arm and forearm anatomy. A 4-dof lightweight, compact, low friction passive mechanism was

designed to allow the alignment between the actuated joint axis rotation and that of the user elbow articulation. A remotely located muscle-like powering unit, consisting of two

antagonist actuators, each composed by a contractile element (hydraulic piston) in series with a non-linear elastic element.
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011 Page 11

THE NEURO-ROBOTICS PARADIGN

The bio-inspired actuation system allows to control the joint position and hardware stiffness by means of an EPH based controller. The NEUROExos elbow module (see the following subsystems: links and the an adaptive mechanical structure, including the double shell structured 4-dof passive mechanism Two remote antagonist muscle-like actuators, powering the elbow module
Figure 4.1)

is a mechatronic platform constituted by

joint by means of steel wire ropes and Bowden cables based transmission. The sensory apparatus, including custom driving cables force sensors. The EPH based controller, acting on the piston position.

The NEUROExos platform is a neuro-robot because it has bio-inspired morphological characteristics (e.g. antagonist muscle-like actuators), and it exploits motion control strategies that are taken from neuroscientific motion control hypothesis (i.e. EPH based control).

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 12

THE NEURO-ROBOTICS PARADIGN

B. Handexos

Figure 4.2: HANDEXOS concept.

To design a wearable mechanism compliant to the human hand movement is a great challenge because of the complexity of the hand structure. This is the reason why one of the main limits of hand exoskeletons is the high level of complexity of the structure and mechanism that often cause low aesthetic acceptability, large overall size and large weight of the device. Analyzing state of the art of recently developed exoskeletons, we can notice a clear trend in trying to overcome the complexity, but as a consequence of this, the number of the controllable DoFs decreases. The design goal of the hand exoskeleton that we are developing (HANDEXOS) is to match the two opposing requirements of allowing free hand motion but keeping low the complexity of the mechanism. The mechanical design of HANDEXOS follows some important criteria: 5 independent finger modules (see Figure 4.1 and Figure 4.2). Full mobility of the hand with a natural range of motion.
Page 13

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

THE NEURO-ROBOTICS PARADIGN

Axes of rotation of the exoskeleton joints being constantly aligned with

that of the finger. Passive and adjustable mechanism on the intermediate phalanx to fit as

much as possible over hands of different sizes. Compact, lightweight and low inertia both on the lateral side of the fingers

and on the upper an lower side of the hand to allow easy wearability. Remote actuation system in order to obtain lightweight mechanism. Palm area and each fingertip free in order to enable the subject to interact

with objects during rehabilitative practice as required when therapy exploits tactile feedback.\ In particular, the Distal Interphalangeal and the Proximal interphalangeal joints are coupled with a coincident revolute joint on the robot, while the Metacarpophalangeal joint was coupled with a 2 DoF mechanism, consisting of a prismatic and a revolute joint, allowing the constant alignment. HANDEXOS has been designed in order to implement different actuation/transmission solutions: underactuation with linear springs. Underactuation with non-linear springs. Independent actuation with non linear springs for each joint.

Both underactuation and non linear springs are concepts derived from bio-inspired design. The first one mimics the configuration of the Extensor Digitorum Profundus in the human hand allowing to have lower number of actuators than DoFs, so that the need of full mobility of the wearers hand can match the requirements of low size and weight. As a drawback, it is not possible to independently control the stiffness and position of each joint. The non linear springs allow to mimic the force elongation characteristic of the human muscle-tendon complex. This, along with an antagonistic configuration, allows to simultaneously and independently control the joint angular position and stiffness. On the other side, this approach requires a high number of actuators (two for each joint), increasing the overall size and
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011 Page 14

THE NEURO-ROBOTICS PARADIGN

weight of the actuation block, which represents a great drawback in a portable system. At the moment the first two actuation strategies have been explored, and planned work will be to study the underactuation with non linear springs configuration in order to exploit the advantages coming from both the actuation strategies.

Figure 3.3: Overview of the HANDEXOS index finger module

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 15

THE NEURO-ROBOTICS PARADIGN

5. CONCLUSIONS
Three innovative neurorobots were presented in this paper as case studies of novel design approach to go beyond limitations of state of the art robotsThe NEURARM, NEUROExos, and HANDEXOS are presented and analysed in order to show ongoing effort in developing novel bioinspired control strategies and actutator systems that can be exploited in wearable robots.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, SEPTEMBER 2011

Page 16