Papers by Roderic A. Grupen
AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN ... more AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN B.E., MADURAI-KAMARAJ UNIVERSITY, INDIA M.Sc.(Engg.), INDIAN INSTITUTE OF SCIENCE, BANGALORE, INDIA Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Andrew G. Barto To operate e#ectively in complex environments learning agents require the ability to form useful abstractions, that is, the ability to selectively ignore irrelevant details. Stated in general terms this is a very di#cult problem. Much of the work in this field is specialized to specific modeling paradigms or classes of problems. In this thesis we introduce an abstraction framework for Markov decision processes (MDPs) based on homomorphisms relating MDPs. We build on classical finite-state automata literature and develop a minimization framework for MDPs that can exploit structure and symmetries to derive smaller equivalent models of the problem. Since employing homomorphisms for minimization requires that the...
We present a framework for learning features for visual discrimination. The learning system is ex... more We present a framework for learning features for visual discrimination. The learning system is exposed to a sequence of training images. Whenever it fails to recognize a visual context adequately, new features are sought that discriminate further between the true and false classes. Features consist of hierarchical combinations of primitive features (local edge and texture characteristics) that are sampled from example images. The system continues to learn better features even after all recognition errors have been eliminated, similarly to mechanisms underlying human visual expertise. Whenever the probabilistic recognition algorithm returns any posterior class probabilities greater than zero and less than one, the system attempts to find new features that improve discrimination between the classes in question. Our experiments indicate that this procedure tends to improve classification accuracy on independent test images, while reducing the number of features used for recognition.
In cognitive robotics "ongoing development" refers to the ability to continuously build... more In cognitive robotics "ongoing development" refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics ...
Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113)
Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity o... more Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity of their hands to perceive the environment as well as to manipulate it. This duality makes sense when we observe that natural environments are dynamic, uncertain, and partially observable. To successfully manipulate objects in this setting requires that uncertainties, specifically those that can cause catastrophic outcomes, are actively suppressed. We expect these fundamental challenges are not specific to biological systems and evolution, but are instead fundamental properties of the manipulation domain that arise due to the wide range of sensitivity that manual interactions can have to context. We propose that these observations suggest that the robotic analogs of manipulation behavior in the animal kingdom must actively control uncertainty in a manner that is objectand context-dependent and, thus, that balances the risk of acting against the cost of gathering additional information fro...
2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005
In robot and sensor networks, one of the key challenges is to decide when and where to deploy sen... more In robot and sensor networks, one of the key challenges is to decide when and where to deploy sensory resources to gather information of optimal value. The problem is essentially one of planning, scheduling and controlling the sensors in the network to acquire data from an environment that is constantly varying. The dynamic nature of the problem precludes the use of traditional rule-based strategies that can handle only quasi-static context changes. Automatic context derivation procedures are thus essential for providing fault recovery and fault pre-emption in such systems. We posit that the quality of a sensor network configuration depends on sensor coverage and geometry, sensor allocation policies, and the dynamic processes in the environment. In this paper, we show how these factors can be manipulated in an adaptive framework for robust run-time resource management. We demonstrate our ideas in a people tracking application using a network of multiple cameras. The task specification for our multi-camera network is one of allocating a camera pair that can best localize a human subject given the current context. The system automatically derives policies for switching between camera pairs that enable robust tracking while being attentive to performance measures. Our approach is unique in that we do not make any a priori assumptions about the scene or the activities that take place in the scene. Models of motion dynamics in the scene and the camera network configuration steer the policies to provide robust tracking.
Abstract: This paper describes the design and preliminary implementation of two distributed smart... more Abstract: This paper describes the design and preliminary implementation of two distributed smart camera applications: a fall detector and an object finder. These functions are part of a novel suite of applications being developed to address aging in place health care ...
ABSTRACT Contents 1 Overview 3 1.1 The Center Structure : : : : : : : : : : : : : : : : : : : : :... more ABSTRACT Contents 1 Overview 3 1.1 The Center Structure : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 3 1.2 CARTS - Center for Autonomous, Real-Time Systems : : : : : : : : : : : : : : : : : 3 1.3 Automated Assembly Application : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 2 CARTS Research Results 5 2.1 Acquiring Geometric Information : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 2.1.1 Interpreting Kinesthetic Information : : : : : : : : : : : : : : : : : : : : : : 5 2.1.2 Animate Vision for Visual Surface Reconstruction : : : : : : : : : : : : : : : 6 2.1.3 Model Extension : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 2.2 Sensor-Based Navigation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 2.2.1 Qualitative Navigation via Image-Based Homing<F4.
This paper describes a representation of the meanings of verbs based on the dynamics of interacti... more This paper describes a representation of the meanings of verbs based on the dynamics of interactions between two agents or objects. The representation treats interactions as having three phases, before, during and after contact. Maps for these phases are constructed. Trajectories through these maps correspond to different types of interactions and are denoted by different verbs. We summarize the results of experiments on learning and reasoning with maps.
Proceedings of the AAAI Conference on Artificial Intelligence
We describe a robot system that autonomously acquires skills through interaction with its environ... more We describe a robot system that autonomously acquires skills through interaction with its environment. The robot learns to sequence the execution of a set of innate controllers to solve a task, extracts and retains components of that solution as portable skills, and then transfers those skills to reduce the time required to learn to solve a second task.
Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006.
Most mobile robots in existence today have large, heavy, statically stable bases. To enable manip... more Most mobile robots in existence today have large, heavy, statically stable bases. To enable manipulation, actuated arms and effectors are attached to the base providing limited postural control. On the other hand, some of the latest developments in humanoid robotics have succeeded in reducing the footprint and raising the center of mass of a robot using whole body postural control and balancing. By comparing the magnitude of various forces that can be applied to the environment, we demonstrate the advantages of whole body postural control. The results presented suggest that for pushing, pulling, or carrying types of tasks, using whole body postural control can lead to higher performance by allowing a platform to apply more force to the environment.
In cognitive robotics “ongoing development” refers to the ability to continuously build on what t... more In cognitive robotics “ongoing development” refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics and infant...
Applications involving teams of mobile robots will require robots within the system to form conne... more Applications involving teams of mobile robots will require robots within the system to form connections to other members with certain quality of service (QoS) requirements. We present a distributed control architecture that allows robots participating in routing a QoS flow to maintain the required level of service while addressing secondary objectives. A distributed control system preserves global properties using "best-effort", error-suppressing controllers. We outline a routing protocol that dynamically reconfigures a flow by recruiting neighboring robots if it believes a routing fault may occur. We evaluate the control architecture and dynamic configuration protocol in simulation, using the ns-2 network simulator and Player/Stage robot simulator platforms.
The UMASS uBot-4 is a two wheeled, dynamically stable, bimanual mobile manipulator. It is a compa... more The UMASS uBot-4 is a two wheeled, dynamically stable, bimanual mobile manipulator. It is a compact, safe, and cost effective platform with many features such as whole body postural control, force sensing actuators, two 4-DOF arms, and a small footprint. It is the latest in a series of small mobile robots that originated with the uBot-0.5 (ca. 1997). This paper presents the motivations for the design of the uBot series and describes how the platform evolved from a small reactive heat-seeking robot to a highly capable mobile manipulator.
2007 7th IEEE-RAS International Conference on Humanoid Robots, 2007
This paper presents a hierarchical, statistical topic model for representing the grasp preshapes ... more This paper presents a hierarchical, statistical topic model for representing the grasp preshapes of a set of objects. Observations provided by teleoperation are clustered into latent affordances shared among all objects. Each affordance defines a joint distribution over position and orientation of the hand relative to the object and conditioned on visual appearance. The parameters of the model are learned using a Gibbs sampling method. After training, the model can be used to compute grasp preshapes for a novel object based on its visual appearance. The model is evaluated experimentally on a set of objects for its ability to generate grasp preshapes that lead to successful grasps, and compared to a baseline approach.
Robotics: Science and Systems VIII, 2012
We present a new Bayesian policy search algorithm suitable for problems with policy-dependent cos... more We present a new Bayesian policy search algorithm suitable for problems with policy-dependent cost variance, a property present in many robot control tasks. We extend recent work on variational heteroscedastic Gaussian processes to the optimization case to achieve efficient minimization of very noisy cost signals. In contrast to most policy search algorithms, our method explicitly models the cost variance in regions of low expected cost and permits runtime adjustment of risk sensitivity without relearning. Our experiments with artificial systems and a real mobile manipulator demonstrate that flexible risk-sensitive policies can be learned in very few trials.
2011 11th IEEE-RAS International Conference on Humanoid Robots, 2011
The biomechanics community has recently made progress toward understanding the role of rapid arm ... more The biomechanics community has recently made progress toward understanding the role of rapid arm movements in human stability recovery. However, comparatively little work has been done exploring this type of control in humanoid robots. We provide a summary of recent insights into the functional contributions of arm recovery motions in humans and experimentally demonstrate advantages of this behavior on a dynamically stable mobile manipulator. Using Bayesian optimization, the robot efficiently discovers policies that reduce total energy expenditure and recovery footprint, and increase ability to stabilize after large impacts.
Proceedings of the 10th international conference on Intelligent user interfaces, 2005
We describe a method for predicting user intentions as part of a human-robot interface. In partic... more We describe a method for predicting user intentions as part of a human-robot interface. In particular, we show that funnels, i.e., geometric objects that partition the input space, provide a convenient means for discriminating individual objects and for clustering sets of objects for hierarchical tasks. One advantage of the proposed implementation is that very few parameters need tuning, and a simple heuristic for setting initial parameter values appears promising. We discuss the possibility of adapting the user interface with machine learning techniques, and we illustrate the approach with a humanoid robot performing a variation of a standard peg-insertion task.
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011
In this paper, a multi-agent search scheme is presented that supports the recognition of activiti... more In this paper, a multi-agent search scheme is presented that supports the recognition of activities and, thus, learning methods for cooperative human-robot interaction. In our approach, stochastic models of human search activity are used to estimate state for HRI. The robot updates a Probabilistic Distribution Function of the target object using the observations and the estimated state of human peers. By this means the robot can choose places to search to compensate the behavior of human peers. This paper also presents an implicit interface design for robot assisted tasks, which allows the robot to infer the intention of the user and to provide assistance autonomously. It reduces the cognitive workload of the user and therefore is useful for elder care applications. The effectiveness and the efficiency of the proposed approaches are demonstrated in the experimental results.
Proceedings of the 6th international conference on Human-robot interaction - HRI '11, 2011
In this case study, we examine the functional utility of an embodied agent as an interactive medi... more In this case study, we examine the functional utility of an embodied agent as an interactive medium in stroke rehab. A set of physical rehab exercises is conducted through the direct engagement of an embodied agent, the uBot-5. Based on the preliminary data, we argue that a general-purpose embodied agent has a potential to functionally complement human therapists in providing rehab to stroke patients.
Uploads
Papers by Roderic A. Grupen