2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 2018
Little is known about the computational mechanisms of how imitation skills develop along with inf... more Little is known about the computational mechanisms of how imitation skills develop along with infant sensorimotor learning. In robotics, there are several well developed frameworks for imitation learning or so called learning by demonstration. Two paradigms dominate: Direct Learning (DL) and Inverse Reinforcement Learning (IRL). The former is a simple mechanism where the observed state and action pairs are associated to construct a copy of the action policy of the demonstrator. In the latter, an optimality principle or reward structure is sought that would explain the observed behavior as the optimal solution governed by the optimality principle or the reward function found. In this study, we explore the plausibility of whether some form of IRL mechanism in infants can facilitate imitation learning and understanding of others' behaviours. We propose that infants project the events taking place in the environment into their internal representations through a set of features that evolve during development. We implement this idea on a grid world environment, which can be considered as a simple model for reaching with obstacle avoidance. The observing infant has to imitate the demonstrator's reaching behavior through IRL by using various set of features that correspond to different stages of development. Our simulation results indicate that the U-shape performance change during imitation development observed in infants can be reproduced with the proposed model.
When interacting with infants, human adults modify their behaviours in an exaggerated manner. Pre... more When interacting with infants, human adults modify their behaviours in an exaggerated manner. Previous studies have demonstrated that infant-directed modification affects the infant's behaviour. However, little is known about how infant-directed modification is elicited during infant-parent interaction. We investigated whether and how the infant's behaviour affects the mother's action during an interaction. We recorded three-dimensional information of cup movements while mothers demonstrated a cup-nesting task during interaction with their infants aged 11 to 13 months. Analyses revealed that spatial characteristics of the mother's task demonstration clearly changed depending on the infant's object manipulation. In particular, the variance in the distance that the cup was moved decreased after the infant's cup nesting and increased after the infant's task-irrelevant manipulation (e.g. cup banging). This pattern was not observed for mothers with 6- to 8-mon...
2008 7th IEEE International Conference on Development and Learning, 2008
Parents significantly alter their infant-directed actions compared to adult-directed ones, which ... more Parents significantly alter their infant-directed actions compared to adult-directed ones, which is assumed to assist the infants' processing of the actions. This paper discusses differences in parental action modification depending on whether the goal or the means is more crucial. When demonstrating a task to an infant, parents try to emphasize the important aspects of the task by suppressing or adding their movement. Our hypothesis is that in a goal-crucial task, the initial and final states of the task should be highlighted by parental actions, whereas in a means-crucial task the movement is underlined. Our analysis using a saliency-based attention model partially verified it: When focusing on the goal, parents tended to emphasize the initial and final states of the objects used in the task by taking a long pause before/after they started/fulfilled the task. When focusing on the means, parents shook the object to highlight it, which consequently made its state invisible. We discuss our findings regarding the uniqueness and commonality of the parental action modification. We also describe our contribution to the development of robots capable of imitating human actions.
2008 IEEE International Conference on Robotics and Automation, 2008
How to teach actions to a robot as well as how a robot learns actions is an important issue to be... more How to teach actions to a robot as well as how a robot learns actions is an important issue to be discussed in designing robot learning systems. Inspired by human parentinfant interaction, we hypothesize that a robot equipped with infant-like abilities can take advantage of parental proper teaching. Parents are known to significantly alter their infantdirected actions versus adult-directed ones, e.g. make more pauses between movements, which is assumed to aid the infants' understanding of the actions. As a first step, we analyzed parental actions using a primal attention model. The model based on visual saliency can detect likely important locations in a scene without employing any knowledge about the actions or the environment. Our statistical analysis revealed that the model was able to extract meaningful structures of the actions, e.g. the initial and final state of the actions and the significant state changes in them, which were highlighted by parental action modifications. We further discuss the issue of designing an infant-like robot that can induce parent-like teaching, and present a human-robot interaction experiment evaluating our robot simulation equipped with the saliency model.
The International Journal of Robotics Research, 2017
The promise of robots assisting humans in everyday tasks has led to a variety of research questio... more The promise of robots assisting humans in everyday tasks has led to a variety of research questions and challenges in human-robot collaboration. Here, we address the question of whether and when a robot should take initiative during joint human-robot task execution. We designed a robotic system capable of autonomously performing table-top manipulation tasks while monitoring the environmental state. Our system is able to predict future environmental states and the robot's actions to reach them using a dynamic Bayesian network. To evaluate our system, we implemented three different initiative conditions to trigger the robot's actions. Human-initiated help gives control of the robot action timing to the user; robotinitiated reactive help triggers robot assistance when it detects that the human needs help; robot-initiated proactive help makes the robot help whenever it can. We performed a user study (N=18) to compare the trigger mechanisms in terms of quality of interaction, system performance and perceived sociality of the robot. We found that people collaborate best with a proactive robot, yielding better team fluency and high subjective ratings. However, they prefer having control of when the robot should help, rather than working with a reactive robot that only helps when needed. We also found that participants gazed at the robot's face more during the human-initiated help compared to the other conditions. This shows that asking for the robot's help may lead to a more "social" interaction, without improving the quality of interaction or the system performance.
ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005.
In this paper, I discuss how visual information about deictic gestures influences learning that e... more In this paper, I discuss how visual information about deictic gestures influences learning that enables these gestures to be comprehended. It has been suggested that the ability of human infants to comprehend deictic gestures depends on the physical appearance of gestures, the movement of gestures, and the distance between gestures and the indicated targets. To understand the mechanisms involved, I apply a model that enables a robot to recognize the static orientation of a gesture as an edge image and movement as optical flow. Experiments using a robot reveal that (1) learning to comprehend reaching gestures with all fingers extended is more accelerated than learning to comprehend pointing with the index finger, and that (2) downward tapping movement facilitates learning more than pointing movement along the direction of the gesture. These results suggest that (1) the quantitative difference in the edge features of reaching and pointing that correspond to the directions of gestures influences learning speed, and that (2) the optical flow of tapping movement that offers qualitatively different information from that provided by the edge image makes learning easier than the optical flow of pointing movement.
Robots are less and less programmed to execute a specic behavior, but develop abilities through t... more Robots are less and less programmed to execute a specic behavior, but develop abilities through the interactions with their environment. In our previous studies, we proposed a robotic model for the emergence of helping behavior based on the minimization of the prediction-error. Our hypoth- esis, dierent from traditional emotion contagion models, suggests that minimizing the dierence (or prediction-error) between the prediction of others' future action and the cur- rent observation can motivate infants to help others. Despite promising results, we observed that the prediction of others' actions generated strong perspective dierences, which ul- timately diminished the helping performance of our robotic system. To solve this issue, we propose to predict the eects of actions instead of predicting the actions per se. Such an ability to predict the environmental state has been observed in young infants and seems promising to improve the per- formance of our robotic system.
BACKGROUND Atypical sensory behavior disrupts behavioral adaptation in children with autism spect... more BACKGROUND Atypical sensory behavior disrupts behavioral adaptation in children with autism spectrum disorder (ASD); however, neural correlates of sensory dysfunction using magnetoencephalography (MEG) remain unclear. METHOD We used MEG to measure the cortical activation elicited by visual (uni)/audiovisual (multisensory) movies in 46 children (7-14 years) were included in final analysis: 13 boys with atypical audiovisual behavior in ASD (AAV+), 10 without this condition, and 23 age-matched typically developing boys. RESULTS The AAV+ group demonstrated an increase in the cortical activation in the bilateral insula in response to unisensory movies and in the left occipital, right superior temporal sulcus (rSTS), and temporal regions to multisensory movies. These increased responses were correlated with severity of the sensory impairment. Increased theta-low gamma oscillations were observed in the rSTS in AAV+. CONCLUSION The findings suggest that AAV is attributed to atypical neural networks centered on the rSTS.
IEEE Transactions on Autonomous Mental Development
A difficulty in robot action learning is that robots do not know where to attend when observing a... more A difficulty in robot action learning is that robots do not know where to attend when observing action demonstration. Inspired by human parent-infant interaction, we suggest that parental action demonstration to infants, called motionese, can scaffold robot learning as well as infants'. Since infants' knowledge about the context is limited, which is comparable to robots, parents are supposed to properly guide their attention by emphasizing the important aspects of the action. Our analysis employing a bottom-up attention model revealed that motionese has the effects of highlighting the initial and final states of the action, indicating significant state changes in it, and underlining the properties of objects used in the action. Suppression and addition of parents' body movement and their frequent social signals to infants produced these effects. Our findings are discussed toward designing robots that can take advantage of parental teaching.
The 7th International Conference on Epigenetic Robotics, 2007
This paper presents a new insight from a computational analysis of parental actions. Developmenta... more This paper presents a new insight from a computational analysis of parental actions. Developmental behavioral studies have suggested that parental modifications in their actions directed to infants versus to adults may aid the infants' processing of the actions. We have been analyzing parental actions using a bottom-up attention model so as to take the advantage in robot action learning. Our latest result indicates that parental social signals can be used for a robot to detect significant state changes in the demonstrated action.
The Workshop on Intermodal Action Structuring, 2008
ABSTRACT Motionese is parental action modification directed to infants versus to adults. Parents ... more ABSTRACT Motionese is parental action modification directed to infants versus to adults. Parents enhance the relevant features in their actions so as to maintain the infants' attention and to aid the infants' processing of the actions. This talk presents our computational analysis on parental actions focusing on the visual struc- turing of them. In order to reveal how parents visually modify their actions and how it can scaold robot's learning as well as infants', we applied a bottom-up visual attention model to the analysis of motionese. The model based on saliency can detect conspicuous locations in a scene in terms of primitive features. That is, it can demonstrate what information can be detected by infants and robots even if they are supposed to have no a priori knowledge about the actions. Our analysis using the saliency model revealed that motionese has the eects of (a) highlighting the initial and final state of objects used in the action, (b) underlining the properties of the objects, and (c) indicating the significant state changes in the action. Parents physically emphasized these aspects by suppressing or adding their body movement. Our further analysis comparing dierent actions uncovered the commonality and uniqueness of motionese depending on the action. The talk also presents our robotics approach to designing an infant-like robot that can induce parent-like teaching of human partners. Open issues in robot action learning are that a robot does not know where to attend when observing an action demonstration, and that only the robot designers can properly teach it actions. We therefore developed a robot simulation equipped with the saliency-based attention model and investigated whether it could encourage naive partners to properly teach it as parents do to infants. Our analysis on the human-robot interaction experiment showed that the robot was accepted as an infant-like social agent.
2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2013
Emotion is one of the important elements for humans to communicate with others. Humans are known ... more Emotion is one of the important elements for humans to communicate with others. Humans are known to share basic emotions such as joy and anger although their developmental changes have been studied less. We propose a computational model for the emotional development in infancy. Our model reproduces the differentiation of emotion from pleasant/unpleasant states to six basic emotions as known in psychological studies. The key idea is twofold: the tactile dominance in infant-caregiver interaction and the inherent ability of tactile sense to discriminate pleasant/unpleasant states. Our model consists of probabilistic neural networks called Restricted Boltzmann Machines. The networks are hierarchically organized to first extract important features from tactile, auditory, and visual stimuli and then to integrate them to represent an emotional state. Pleasant/unpleasant information is directly provided to the highest level of the network to facilitate the emotional differentiation. Experimental results show that our model with the tactile dominance leads to better differentiation of emotion than others without such dominance.
2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2013
Before babies acquire an adult-like visual capacity, they participate in a social world as a huma... more Before babies acquire an adult-like visual capacity, they participate in a social world as a human learning system which promotes social activities around them and in turn dramatically alters their own social participation. Visual input becomes more dynamic as they gain self-generated movement, and such movement has a potential role in learning. The present study specifically looks at the expected change in motion of the early visual input that infants are exposed to, and the corresponding attentional coordination within the specific context of parentinfant interactions. The results will be discussed in terms of the significance of social input for development.
2009 IEEE 8th International Conference on Development and Learning, 2009
This research addresses the challenge of developing an action learning model employing bottom-up ... more This research addresses the challenge of developing an action learning model employing bottom-up visual attention. Although bottom-up attention enables robots to autonomously explore the environment, learn to recognize objects, and interact with humans, the instability of their attention as well as the poor quality of the information detected at the attentional location has hindered the robots from processing dynamic movements. In order to learn actions, robots have to stably attend to the relevant movement by ignoring noises while maintaining sensitivity to a new important movement. To meet these contradictory requirements, I introduce mechanisms for retinal filtering and stochastic attention selection inspired by human vision. The former reduces the complexity of the peripheral vision and thus enables robots to focus more on the currently-attended location. The latter allows robots to flexibly shift their attention to a new prominent location, which must be relevant to the demonstrated action. The signals detected at the attentional location are then enriched based on the spatial and temporal continuity so that robots can learn to recognize objects, movements, and their associations. Experimental results show that the proposed system can extract key actions from human action demonstrations.
An open question in robot action learning is how robots can detect relevant features of demonstra... more An open question in robot action learning is how robots can detect relevant features of demonstrated actions. Robots which have no knowledge about the action nor the environment encounter the problem of not knowing what to detect and where to attend. Inspired by human parent-infant interaction, we suggest that parental action to infants can assist robots as well as infants detecting important aspects of the action. Parents are known to significantly alter their infant-directed action versus adult-directed one. They, for example, exaggerate their body movement and make more pauses between movements. Our hypothesis is that parental action modification such as suppression and addition of their body movement physically highlights the important aspects of the action, so that they can be detected by bottom-up attention. The talk first presents our analytical experiment focusing on parent-infant interaction [1]. We found that parental action emphasized the initial and final states of a task, significant events in it, and the properties of objects used in it, which were extracted by a bottom-up attention model based on saliency. The talk then presents our human-robot interaction experiment using a robot simulation equipped with the saliency model [2]. The robot's attention was controlled so as to gaze at the most salient location in the interaction. Our qualitative analysis on people's reaction revealed that the saliency model enabled our robot to be accepted as an infant-like agent and therefore induce parent-like proper teaching of the partners. The talk will conclude with our suggestion that the bottom-up attention is a key for robots to shape the interaction and take advantage of human scaffolding for action learning [3].
2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009
This paper presents an architecture extending bottom-up visual attention for dynamic scene analys... more This paper presents an architecture extending bottom-up visual attention for dynamic scene analysis. In dynamic scenes, particularly when learning actions from demonstrations, robots have to stably focus on the relevant movement by disregarding surrounding noises, but still maintain sensitivity to a new relevant movement, which might occur in the surroundings. In order to meet the contradictory requirements of stability and sensitivity for attention, this paper introduces biologically-inspired mechanisms for retinal filtering and stochastic attention selection. The former reduces the complexity of peripheral signals by filtering an input image. It results in enhancing bottom-up saliency in the fovea as well as in detecting only prominent signals from the periphery. The latter allows robots to shift attention to a less but still salient location in the periphery, which is likely relevant to the demonstrated action. Integrating these mechanisms with computation for bottom-up saliency enables robots to extract important action sequences from task demonstrations. Experiments with a simulated and a natural scene show better performance of the proposed model than comparative models.
2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2012
This paper presents a computational approach to measuring information transfer in infant-caregive... more This paper presents a computational approach to measuring information transfer in infant-caregiver interaction. It is supposed that both an infant and a caregiver mutually shape the interaction by sending various signals to each other, of which the dynamic structure changes according to the infant's age. We investigated such developmental change both in infants and caregivers by measuring transfer entropy within and between their body movements. Our analysis demonstrated that infants significantly improve their body coordination and social contingency between 6 to 13 months of age. Their gaze, for example, start responding to caregivers' gaze, which indicates the development of joint attention. The coordination between infants' two hands also drastically improves as they grow. Of particular interest is that such development in infants elicits caregivers' adaptation. Caregivers change their social responses and body coordination to a more sophisticated manner in order to further facilitate infant development. Our approach is the first study to quantitatively verify the "co-development" of infants and caregivers, which appears as increases in information transfer within and between their behaviors.
2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 2018
Little is known about the computational mechanisms of how imitation skills develop along with inf... more Little is known about the computational mechanisms of how imitation skills develop along with infant sensorimotor learning. In robotics, there are several well developed frameworks for imitation learning or so called learning by demonstration. Two paradigms dominate: Direct Learning (DL) and Inverse Reinforcement Learning (IRL). The former is a simple mechanism where the observed state and action pairs are associated to construct a copy of the action policy of the demonstrator. In the latter, an optimality principle or reward structure is sought that would explain the observed behavior as the optimal solution governed by the optimality principle or the reward function found. In this study, we explore the plausibility of whether some form of IRL mechanism in infants can facilitate imitation learning and understanding of others' behaviours. We propose that infants project the events taking place in the environment into their internal representations through a set of features that evolve during development. We implement this idea on a grid world environment, which can be considered as a simple model for reaching with obstacle avoidance. The observing infant has to imitate the demonstrator's reaching behavior through IRL by using various set of features that correspond to different stages of development. Our simulation results indicate that the U-shape performance change during imitation development observed in infants can be reproduced with the proposed model.
When interacting with infants, human adults modify their behaviours in an exaggerated manner. Pre... more When interacting with infants, human adults modify their behaviours in an exaggerated manner. Previous studies have demonstrated that infant-directed modification affects the infant's behaviour. However, little is known about how infant-directed modification is elicited during infant-parent interaction. We investigated whether and how the infant's behaviour affects the mother's action during an interaction. We recorded three-dimensional information of cup movements while mothers demonstrated a cup-nesting task during interaction with their infants aged 11 to 13 months. Analyses revealed that spatial characteristics of the mother's task demonstration clearly changed depending on the infant's object manipulation. In particular, the variance in the distance that the cup was moved decreased after the infant's cup nesting and increased after the infant's task-irrelevant manipulation (e.g. cup banging). This pattern was not observed for mothers with 6- to 8-mon...
2008 7th IEEE International Conference on Development and Learning, 2008
Parents significantly alter their infant-directed actions compared to adult-directed ones, which ... more Parents significantly alter their infant-directed actions compared to adult-directed ones, which is assumed to assist the infants' processing of the actions. This paper discusses differences in parental action modification depending on whether the goal or the means is more crucial. When demonstrating a task to an infant, parents try to emphasize the important aspects of the task by suppressing or adding their movement. Our hypothesis is that in a goal-crucial task, the initial and final states of the task should be highlighted by parental actions, whereas in a means-crucial task the movement is underlined. Our analysis using a saliency-based attention model partially verified it: When focusing on the goal, parents tended to emphasize the initial and final states of the objects used in the task by taking a long pause before/after they started/fulfilled the task. When focusing on the means, parents shook the object to highlight it, which consequently made its state invisible. We discuss our findings regarding the uniqueness and commonality of the parental action modification. We also describe our contribution to the development of robots capable of imitating human actions.
2008 IEEE International Conference on Robotics and Automation, 2008
How to teach actions to a robot as well as how a robot learns actions is an important issue to be... more How to teach actions to a robot as well as how a robot learns actions is an important issue to be discussed in designing robot learning systems. Inspired by human parentinfant interaction, we hypothesize that a robot equipped with infant-like abilities can take advantage of parental proper teaching. Parents are known to significantly alter their infantdirected actions versus adult-directed ones, e.g. make more pauses between movements, which is assumed to aid the infants' understanding of the actions. As a first step, we analyzed parental actions using a primal attention model. The model based on visual saliency can detect likely important locations in a scene without employing any knowledge about the actions or the environment. Our statistical analysis revealed that the model was able to extract meaningful structures of the actions, e.g. the initial and final state of the actions and the significant state changes in them, which were highlighted by parental action modifications. We further discuss the issue of designing an infant-like robot that can induce parent-like teaching, and present a human-robot interaction experiment evaluating our robot simulation equipped with the saliency model.
The International Journal of Robotics Research, 2017
The promise of robots assisting humans in everyday tasks has led to a variety of research questio... more The promise of robots assisting humans in everyday tasks has led to a variety of research questions and challenges in human-robot collaboration. Here, we address the question of whether and when a robot should take initiative during joint human-robot task execution. We designed a robotic system capable of autonomously performing table-top manipulation tasks while monitoring the environmental state. Our system is able to predict future environmental states and the robot's actions to reach them using a dynamic Bayesian network. To evaluate our system, we implemented three different initiative conditions to trigger the robot's actions. Human-initiated help gives control of the robot action timing to the user; robotinitiated reactive help triggers robot assistance when it detects that the human needs help; robot-initiated proactive help makes the robot help whenever it can. We performed a user study (N=18) to compare the trigger mechanisms in terms of quality of interaction, system performance and perceived sociality of the robot. We found that people collaborate best with a proactive robot, yielding better team fluency and high subjective ratings. However, they prefer having control of when the robot should help, rather than working with a reactive robot that only helps when needed. We also found that participants gazed at the robot's face more during the human-initiated help compared to the other conditions. This shows that asking for the robot's help may lead to a more "social" interaction, without improving the quality of interaction or the system performance.
ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005.
In this paper, I discuss how visual information about deictic gestures influences learning that e... more In this paper, I discuss how visual information about deictic gestures influences learning that enables these gestures to be comprehended. It has been suggested that the ability of human infants to comprehend deictic gestures depends on the physical appearance of gestures, the movement of gestures, and the distance between gestures and the indicated targets. To understand the mechanisms involved, I apply a model that enables a robot to recognize the static orientation of a gesture as an edge image and movement as optical flow. Experiments using a robot reveal that (1) learning to comprehend reaching gestures with all fingers extended is more accelerated than learning to comprehend pointing with the index finger, and that (2) downward tapping movement facilitates learning more than pointing movement along the direction of the gesture. These results suggest that (1) the quantitative difference in the edge features of reaching and pointing that correspond to the directions of gestures influences learning speed, and that (2) the optical flow of tapping movement that offers qualitatively different information from that provided by the edge image makes learning easier than the optical flow of pointing movement.
Robots are less and less programmed to execute a specic behavior, but develop abilities through t... more Robots are less and less programmed to execute a specic behavior, but develop abilities through the interactions with their environment. In our previous studies, we proposed a robotic model for the emergence of helping behavior based on the minimization of the prediction-error. Our hypoth- esis, dierent from traditional emotion contagion models, suggests that minimizing the dierence (or prediction-error) between the prediction of others' future action and the cur- rent observation can motivate infants to help others. Despite promising results, we observed that the prediction of others' actions generated strong perspective dierences, which ul- timately diminished the helping performance of our robotic system. To solve this issue, we propose to predict the eects of actions instead of predicting the actions per se. Such an ability to predict the environmental state has been observed in young infants and seems promising to improve the per- formance of our robotic system.
BACKGROUND Atypical sensory behavior disrupts behavioral adaptation in children with autism spect... more BACKGROUND Atypical sensory behavior disrupts behavioral adaptation in children with autism spectrum disorder (ASD); however, neural correlates of sensory dysfunction using magnetoencephalography (MEG) remain unclear. METHOD We used MEG to measure the cortical activation elicited by visual (uni)/audiovisual (multisensory) movies in 46 children (7-14 years) were included in final analysis: 13 boys with atypical audiovisual behavior in ASD (AAV+), 10 without this condition, and 23 age-matched typically developing boys. RESULTS The AAV+ group demonstrated an increase in the cortical activation in the bilateral insula in response to unisensory movies and in the left occipital, right superior temporal sulcus (rSTS), and temporal regions to multisensory movies. These increased responses were correlated with severity of the sensory impairment. Increased theta-low gamma oscillations were observed in the rSTS in AAV+. CONCLUSION The findings suggest that AAV is attributed to atypical neural networks centered on the rSTS.
IEEE Transactions on Autonomous Mental Development
A difficulty in robot action learning is that robots do not know where to attend when observing a... more A difficulty in robot action learning is that robots do not know where to attend when observing action demonstration. Inspired by human parent-infant interaction, we suggest that parental action demonstration to infants, called motionese, can scaffold robot learning as well as infants'. Since infants' knowledge about the context is limited, which is comparable to robots, parents are supposed to properly guide their attention by emphasizing the important aspects of the action. Our analysis employing a bottom-up attention model revealed that motionese has the effects of highlighting the initial and final states of the action, indicating significant state changes in it, and underlining the properties of objects used in the action. Suppression and addition of parents' body movement and their frequent social signals to infants produced these effects. Our findings are discussed toward designing robots that can take advantage of parental teaching.
The 7th International Conference on Epigenetic Robotics, 2007
This paper presents a new insight from a computational analysis of parental actions. Developmenta... more This paper presents a new insight from a computational analysis of parental actions. Developmental behavioral studies have suggested that parental modifications in their actions directed to infants versus to adults may aid the infants' processing of the actions. We have been analyzing parental actions using a bottom-up attention model so as to take the advantage in robot action learning. Our latest result indicates that parental social signals can be used for a robot to detect significant state changes in the demonstrated action.
The Workshop on Intermodal Action Structuring, 2008
ABSTRACT Motionese is parental action modification directed to infants versus to adults. Parents ... more ABSTRACT Motionese is parental action modification directed to infants versus to adults. Parents enhance the relevant features in their actions so as to maintain the infants' attention and to aid the infants' processing of the actions. This talk presents our computational analysis on parental actions focusing on the visual struc- turing of them. In order to reveal how parents visually modify their actions and how it can scaold robot's learning as well as infants', we applied a bottom-up visual attention model to the analysis of motionese. The model based on saliency can detect conspicuous locations in a scene in terms of primitive features. That is, it can demonstrate what information can be detected by infants and robots even if they are supposed to have no a priori knowledge about the actions. Our analysis using the saliency model revealed that motionese has the eects of (a) highlighting the initial and final state of objects used in the action, (b) underlining the properties of the objects, and (c) indicating the significant state changes in the action. Parents physically emphasized these aspects by suppressing or adding their body movement. Our further analysis comparing dierent actions uncovered the commonality and uniqueness of motionese depending on the action. The talk also presents our robotics approach to designing an infant-like robot that can induce parent-like teaching of human partners. Open issues in robot action learning are that a robot does not know where to attend when observing an action demonstration, and that only the robot designers can properly teach it actions. We therefore developed a robot simulation equipped with the saliency-based attention model and investigated whether it could encourage naive partners to properly teach it as parents do to infants. Our analysis on the human-robot interaction experiment showed that the robot was accepted as an infant-like social agent.
2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2013
Emotion is one of the important elements for humans to communicate with others. Humans are known ... more Emotion is one of the important elements for humans to communicate with others. Humans are known to share basic emotions such as joy and anger although their developmental changes have been studied less. We propose a computational model for the emotional development in infancy. Our model reproduces the differentiation of emotion from pleasant/unpleasant states to six basic emotions as known in psychological studies. The key idea is twofold: the tactile dominance in infant-caregiver interaction and the inherent ability of tactile sense to discriminate pleasant/unpleasant states. Our model consists of probabilistic neural networks called Restricted Boltzmann Machines. The networks are hierarchically organized to first extract important features from tactile, auditory, and visual stimuli and then to integrate them to represent an emotional state. Pleasant/unpleasant information is directly provided to the highest level of the network to facilitate the emotional differentiation. Experimental results show that our model with the tactile dominance leads to better differentiation of emotion than others without such dominance.
2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2013
Before babies acquire an adult-like visual capacity, they participate in a social world as a huma... more Before babies acquire an adult-like visual capacity, they participate in a social world as a human learning system which promotes social activities around them and in turn dramatically alters their own social participation. Visual input becomes more dynamic as they gain self-generated movement, and such movement has a potential role in learning. The present study specifically looks at the expected change in motion of the early visual input that infants are exposed to, and the corresponding attentional coordination within the specific context of parentinfant interactions. The results will be discussed in terms of the significance of social input for development.
2009 IEEE 8th International Conference on Development and Learning, 2009
This research addresses the challenge of developing an action learning model employing bottom-up ... more This research addresses the challenge of developing an action learning model employing bottom-up visual attention. Although bottom-up attention enables robots to autonomously explore the environment, learn to recognize objects, and interact with humans, the instability of their attention as well as the poor quality of the information detected at the attentional location has hindered the robots from processing dynamic movements. In order to learn actions, robots have to stably attend to the relevant movement by ignoring noises while maintaining sensitivity to a new important movement. To meet these contradictory requirements, I introduce mechanisms for retinal filtering and stochastic attention selection inspired by human vision. The former reduces the complexity of the peripheral vision and thus enables robots to focus more on the currently-attended location. The latter allows robots to flexibly shift their attention to a new prominent location, which must be relevant to the demonstrated action. The signals detected at the attentional location are then enriched based on the spatial and temporal continuity so that robots can learn to recognize objects, movements, and their associations. Experimental results show that the proposed system can extract key actions from human action demonstrations.
An open question in robot action learning is how robots can detect relevant features of demonstra... more An open question in robot action learning is how robots can detect relevant features of demonstrated actions. Robots which have no knowledge about the action nor the environment encounter the problem of not knowing what to detect and where to attend. Inspired by human parent-infant interaction, we suggest that parental action to infants can assist robots as well as infants detecting important aspects of the action. Parents are known to significantly alter their infant-directed action versus adult-directed one. They, for example, exaggerate their body movement and make more pauses between movements. Our hypothesis is that parental action modification such as suppression and addition of their body movement physically highlights the important aspects of the action, so that they can be detected by bottom-up attention. The talk first presents our analytical experiment focusing on parent-infant interaction [1]. We found that parental action emphasized the initial and final states of a task, significant events in it, and the properties of objects used in it, which were extracted by a bottom-up attention model based on saliency. The talk then presents our human-robot interaction experiment using a robot simulation equipped with the saliency model [2]. The robot's attention was controlled so as to gaze at the most salient location in the interaction. Our qualitative analysis on people's reaction revealed that the saliency model enabled our robot to be accepted as an infant-like agent and therefore induce parent-like proper teaching of the partners. The talk will conclude with our suggestion that the bottom-up attention is a key for robots to shape the interaction and take advantage of human scaffolding for action learning [3].
2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009
This paper presents an architecture extending bottom-up visual attention for dynamic scene analys... more This paper presents an architecture extending bottom-up visual attention for dynamic scene analysis. In dynamic scenes, particularly when learning actions from demonstrations, robots have to stably focus on the relevant movement by disregarding surrounding noises, but still maintain sensitivity to a new relevant movement, which might occur in the surroundings. In order to meet the contradictory requirements of stability and sensitivity for attention, this paper introduces biologically-inspired mechanisms for retinal filtering and stochastic attention selection. The former reduces the complexity of peripheral signals by filtering an input image. It results in enhancing bottom-up saliency in the fovea as well as in detecting only prominent signals from the periphery. The latter allows robots to shift attention to a less but still salient location in the periphery, which is likely relevant to the demonstrated action. Integrating these mechanisms with computation for bottom-up saliency enables robots to extract important action sequences from task demonstrations. Experiments with a simulated and a natural scene show better performance of the proposed model than comparative models.
2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2012
This paper presents a computational approach to measuring information transfer in infant-caregive... more This paper presents a computational approach to measuring information transfer in infant-caregiver interaction. It is supposed that both an infant and a caregiver mutually shape the interaction by sending various signals to each other, of which the dynamic structure changes according to the infant's age. We investigated such developmental change both in infants and caregivers by measuring transfer entropy within and between their body movements. Our analysis demonstrated that infants significantly improve their body coordination and social contingency between 6 to 13 months of age. Their gaze, for example, start responding to caregivers' gaze, which indicates the development of joint attention. The coordination between infants' two hands also drastically improves as they grow. Of particular interest is that such development in infants elicits caregivers' adaptation. Caregivers change their social responses and body coordination to a more sophisticated manner in order to further facilitate infant development. Our approach is the first study to quantitatively verify the "co-development" of infants and caregivers, which appears as increases in information transfer within and between their behaviors.
Uploads
Papers by Yukie Nagai