CIRP Annals - Manufacturing Technology 72 (2023) 1
4

Contents lists available at ScienceDirect

CIRP Annals - Manufacturing Technology

journal homepage: https://www.editorialmanager.com/CIRP/default.aspx

A collaborative intelligence-based approach for handling human-robot

collaboration uncertainties
Pai Zhenga, Shufei Lia, Junming Fana, Chengxi Lia, Lihui Wang (1)b,*
a
Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, HKSAR, China
b
Department of Production Engineering, KTH Royal Institute of Technology, Sweden

A R T I C L E I N F O A B S T R A C T

Article history: Human-Robot Collaboration (HRC) has played a pivotal role in today’s human-centric smart manufacturing sce-
Available online 15 April 2023 narios. Nevertheless, limited concerns have been given to HRC uncertainties. By integrating both human and
artificial intelligence, this paper proposes a Collaborative Intelligence (CI)-based approach for handling three
Keywords: major types of HRC uncertainties (i.e., human, robot and task uncertainties). A fine-grained human digital twin
Human-robot collaboration
modelling method is introduced to address human uncertainties with better robotic assistance. Meanwhile, a
Manufacturing system
Collaborative intelligence
learning from demonstration approach is offered to handle robotic task uncertainties with human intelligence.
Lastly, the feasibility of the proposed CI has been demonstrated in an illustrative HRC assembly task.
© 2023 The Authors. Published by Elsevier Ltd on behalf of CIRP. This is an open access article under the CC BY
license (http://creativecommons.org/licenses/by/4.0/)

1. Introduction To tackle these problems, this paper introduces a Collaborative

Existing automation systems have reached its bottleneck in handling Intelligence (CI)-based approach to handling industrial uncertainties
various flexible manufacturing tasks, such as the assembly or disassem- in HRC systems with respective tolerance settings. According to Wil-
bly of complicated products with frequent changes [1]. To satisfy son and Daugherty [7], CI looks forward to joining forces between
human-centric needs in these working conditions, Human-Robot Col- human intelligence and artificial intelligence from both humans-
laboration (HRC) is becoming an active area that combines the strength, assisting-machines and machines-assisting-humans perspectives. In
repeatability and accuracy of robots with the cognitive flexibility of HRC scenarios, the proposed CI-based approach is mainly embodied
humans to achieve a high-level flexible automation [2]. Especially, in 1) human-assisting-robot, humans train robots how to adjust oper-
recent studies on the proactive HRC approaches immerse mutual-cogni- ations for new situations, explain which decision is taken, and sustain
tive, predictable, and even self-organising elements into a human-robot what knowledge is updated in task and robot uncertainties; and 2)
execution loop [3]. In this context, human and robotic agents can under- robot-assisting-human, robot can amplify human easy working condi-
stand bi-directional operation intention, learn task operation sequences tions, let humans interact with work with flexible decisions, and
and even execute adaptive manipulations in a self-organised manner embody human-centric needs when facing human uncertainties. Fol-
[4]. Nevertheless, many uncertainties still exist in real-life HRC lowing this manner, the rest of this paper is organised as follows.
manufacturing scenarios, which impede their successful and effective First, Section 2 depicts the systematic methodology of CI-based HRC.
implementations. To the authors’ best knowledge, those HRC uncertain- Then, Section 3 gives a demonstrative case study to validate its feasi-
ties can be further classified into three major types, namely human bility. Section 4 finally highlights the conclusions and future works.
uncertainty, task uncertainty and robot uncertainty.
Human uncertainty is the most common one, since humans have a 2. CI-based HRC uncertainties handling
much higher degree of autonomy and spontaneity. It mainly consists
of abnormal human behaviours [5] (e.g., sudden change of gesture, To handle various HRC uncertainties, the proposed CI-based sys-
collisions with the predefined robotic trajectories), which pose a tem framework with core procedures is shown in Fig. 1.
great challenge to the robotic collaborators. Firstly, adaptive HRC task planning can be generated from a holistic
Task uncertainty is also common in many HRC manufacturing sce- scene perception, including object detection, human action recognition
narios, especially in the disassembly/inspection process of used prod- [8], and point cloud segmentation of the surrounding environment. The
ucts, due to their uncertain shapes, sizes and physical conditions, such task planning strategy is denoted by the HRC Knowledge Graph (KG),
as rusted, loosened and defective components, stained surfaces, and which leverages a dynamic graph embedding-based method to infer
geometry changes [6]. human-robot relations and task structures in different stages, achieving
Robot uncertainty denotes its runaway motions in unstructured a mutual understanding of teamwork. In certain situations, the adaptive
workspaces, which may bring confusion in operation arrangement task planning module assigns operations from three aspects, (1) robot-
and human safety issues in human-robot physical interaction. centric mode, (2) human-centric mode, and (3) adaptive mode. The for-
mer two modes grant authorities to robots and humans respectively, so
* Corresponding author. that they can perform different types of tasks based on their own capa-
E-mail address: lihui.wang@iip.kth.se (L. Wang). bilities. Lastly, the adaptive mode allows humans and robots to

https://doi.org/10.1016/j.cirp.2023.04.057
0007-8506/© 2023 The Authors. Published by Elsevier Ltd on behalf of CIRP. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
2 P. Zheng et al. / CIRP Annals - Manufacturing Technology 72 (2023) 1
4

arrangement and sequential orders are obtained by connecting HRTWE

nodes with updated embeddings, achieving HRC KG update. The con-
nection from a robot node to a task node indicates robot action types,
while the following edge from the task node to a workpiece node
denotes the robot endpoints and operation goals. Various task-planning
strategies are represented in an explainable graphical manner.

2.2. Vision-based fine-grained HDT modelling

Based on the adaptive HRC task planning result, the ability to con-
stantly perceive and model the human body is also essential, which
can provide necessary information for the collaborative robot to fur-
ther cope with human related uncertainties during HRC. Previous
endeavours in the HRC literature have been devoted to perceiving
human body skeleton for active collision avoidance [11] or recognis-
ing human action intention [12] for robotic decision-makings. How-
ever, they can only model the human operator in a relatively coarse
manner with insufficient representation fidelity or limited recogni-
tion accuracy. To this end, a vision-based fine-grained HDT modelling
scheme of a human operator is proposed and depicted in Fig. 3, which
mainly consists of two parts: 1) fine-grained human pose reconstruc-
Fig. 1. The proposed CI-based approach for handling HRC uncertainties.
tion, and 2) spatial-temporal human behaviour intention recognition.

coordinately take operations. In uncertain situations, the Human Digital

Twin (HDT) model and robot learning methods are triggered. The HDT
is developed to reconstruct the fine-grained human hand and full body
pose during execution. When a human operator discovers task uncer-
tainties or robot runaway motions, one can teach a robot and update its
manipulation skills from a flexible demonstration via the proposed
Learning from Demonstration (LfD) method. When the HDT captures
human uncertain activities in the task process, the robot can self-opti-
mise its trajectories via a Deep Reinforcement Learning (DRL) based
approach to adapt to humans’ abnormal behaviours for safe and effi-
cient collaboration. Thus, the CI-based HRC maximises human-robot
complementarity and resists uncertain situations in tasks. Fig. 3. Vision-based fine-grained HDT modelling scheme.

2.1. Dynamic graph embedding-based adaptive HRC task planning Fine-grained human pose reconstruction. In the first part of the
HDT, a deep learning model that can simultaneously reconstruct the fine-
Adaptive task planning algorithms free existing HRC systems from grained 3D dense mesh and skeleton joints of the human body is pro-
predefined instructions, for enhanced adaptability in various posed. Concretely, the RGB-D images of the human operator will be proc-
manufacturing scenarios. Our previous works have introduced the scene essed by a ResNet-50 backbone network to extract the geometric
graph (SG) and KG approaches [4] to generate task planning strategies features, which is then utilised to regress the pose parameters
for HRC systems. Meanwhile, Raatz et al. [9] utilised a genetic algorithm ubody 2 R3K , shape parameters bbody 2 RM , 3D rotation Rbody 2 R33 , and
to optimise task scheduling based on capabilities and time assumptions 3D translation Tbody 2 R3 . The pose and shape parameters are subse-
in HRC. However, these methods focus on distilling temporal knowledge quently sent to the SMPL (Skinned Multi-Person Linear model) human
representation of HRC configurations, while ignoring updating the infor- body model
a differentiable function that outputs a triangulated mesh
mation to next-stage task arrangement. Mðu; bÞ 2 R3N . The adoption of the SMPL model can largely simplify the
With perceptual results, a DynGraphGAN model [10] is leveraged to reconstruction process to achieve real-time performance by relying on a
temporally construct and update HRC KG for on-demand task allocation, template human body mesh as a priori which will be bended and
as shown in Fig. 2. Considering input of human behaviours, detected stretched to the target human pose according to the estimated pose and
objects and task structures, there are numerous possible configurations shape parameters. The predicted global 3D rotation and translation will
of their relations. Firstly, a generator generates adjacency matrices to then be applied to obtain the correctly posed human body mesh. The 3D
represent the relation evolution between Human-Robot-Task-Work- skeleton points Xðu; bÞ 2 R3K are further obtained by linear regression
piece-Environment (HRTWE) nodes over time. The updated HRTWE from the final mesh vertices. To further refine hand pose reconstruction
connections may introduce fake edges. Thus, a continuous discriminator of human operators, the ResNet-50 backbone will also regress hand
is leveraged to distinguish the authentic and fake links between HRTWE parameters including pose parameters uhand 2 R3K , shape parameters
nodes via Gated Recurrent Unit (GRU) algorithm. Optimised operation bhand 2 RM , and 3D translation Thand 2 R3 , which will be processed by the
MANO (hand Model with Articulated and Non-rigid defOrmations) to
enhance the SMPL model and to facilitate the human-assisted robot LfD.
Once the fine-grained human mesh is reconstructed, it can represent the
precise geometric occupancy of the human body, which can substantially
reduce the perception error of human body during HRC.
Spatial-temporal human behaviour intention recognition. The
spatial-temporal human behaviour intention estimation amounts to a
higher semantic level, which is essential to complete a holistic HDT in
HRC scenarios. In this module, the RGB video stream and associated
skeleton stream are regarded as the input data source. For the RGB
stream, a 3D ResNet-50 will be used to process the spatial-temporal fea-
tures in a unified convolution structure and extract the image feature.
Fig. 2. Dynamic graph embedding-based adaptive HRC task planning. Meanwhile, the skeleton stream will be split into 4 branches including
 P. Zheng et al. / CIRP Annals - Manufacturing Technology 72 (2023) 1
4 3

head, trunk, arm, and leg branches, each of which will be processed by Algorithm 1
an ST-GCN (Spatial Temporal Graph Convolutional Network) model. Pseudo code of Dagger algorithm.
After the spatial-temporal feature extraction of the local body parts, an Input: Original Dataset D
aggregator network will be utilised to fuse them into the global skeleton Output: Optimal updated policy p ^ update , Aggregated dataset Dagg
feature, which will be fused with the extracted image feature to discrim- Initialisation: Original policy po
1 pi¼1 po
inate the type of current human behaviour and if it is abnormal. Since 2 For episode i = 1, 2, . . ., T do
the model is exclusively trained on normal behaviour data, it will only 3 Sample T
step trajectories with trained  policy pi
4 Generate/Demonstrate dataset Di ¼ f s; p ðsÞg via expert p for unsolved case trial
be able to provide a random guess with extremely low confidence score (determined by expert) by pi
for an unseen abnormal behaviour sequence, of which setting up a tol- 5 Aggregate the Dataset D D [ Di
erance on the confidence scores can eliminate any wrong detection. For 6 Re-train the control policy function piþ1 via behavioural cloning
7 End For
normal human behaviours, the recognised action intention will be 8 Return optimal policy pi determined by evaluating the success rate of task
passed to robots to make action and motion planning ahead of time,
while for abnormal behaviours that cannot be properly parsed by the However, owing to the diversity of samples, the effectiveness of LfD is
system, a warning will be signalled to notify human for behaviour cor- limited by the number and variance amongst expert demonstrations.
rection or optionally trigger the LfD protocol. Therefore, the robot policy that trained by BC algorithm still lacks the
flexibility and adaptability towards newly occurred task or robot uncer-
2.3. Human-assisting-robot via LfD tainties. To address that, an online learning approach with dataset aggre-
Considering robot and task uncertainties, a human operator can gation (Dagger) mechanism is introduced in the LfD process, whose
potentially transfer one’s experience to robot manipulation skills for workflow is shown in Algorithm 1. With the Dagger mechanism, not
flexible and adaptive task execution via the LfD approach. only existing uncertainties but also similar but new ones can be
Human-in-the-loop robot control. To better transfer human addressed. Thereby, the robot can better resolve the dynamic
experts’ engineering practice, the hand pose extracted from HDT is manufacturing tasks with the help of expert intelligence more efficiently.
introduced to implement a seamless hand gesture-enabled robot
control system. As shown in Fig. 4, the method extracts the worker’s 2.4. Robot-assisting-human via DRL
hand poses and corresponds them to the robot end-effector poses, Considering abnormal behaviours caused by human uncertainties,
which could intuitively enable the robot to mimic the hand move- a robot may dynamically re-plan its motion to complete assigned
ment. The worker’s left-hand pose can be accurately extracted via tasks from HRC KG, and also to ensure human safety. A DRL-based
his palm location/orientation and transformed into the end-effector approach is introduced in this work to achieve human uncertainty-
pose of the robot accordingly. aware robot control for safe and adaptive HRC.
The human uncertainty factors detected from HDT are embedded
in the full-body skeleton position, abnormal behaviours warnings,
and intentions. In the implementation, the DRL-approach leverages
uncertainty factors, motion planning success rate, and safe con-
straints in HRC scenarios as the optimisation indexes. Following that,
the robot motion planning process can be formulated as a Markov
Decision Process to optimise the control policy p and reinforce the
Fig. 4. Examples of mapping between human hand and robot end-effector. robot to choose an action at 2 A in the state st 2 S with the largest
cumulated reward. The DRL settings are shown as follows:
Robot learn from demonstration & dataset aggregation. In
Observation state (O) is a state representation of the human-
addition to explicit imitation, the robot movement trajectories dem-
robot working scene, which consists of the human data extracted
onstrated by the experts are recorded as datasets and fed into the LfD
from the abovementioned HDT, including whole-body skeleton posi-
algorithm, a promising learning approach for empowering the skills
tion (P), abnormal behaviours warnings (B), human intentions (I),
of robots. With the dataset, the workers’ patterns could be implicitly
and also robot own state (M). The information can be concatenated in
extracted and drive robots to learn the uncertainties-oriented control
a state vector, O ¼ ðP; B; I; MÞ.
policy for subsequent adaptive robot programs.
Action space (A) refers to the reachability of the robot. In the
In this work, to approximate the control policy function pu ðajsÞ, an LfD
experiment, the inverse kinematics is combined to transform the
approach adopted is Behavioural Cloning (BC) algorithm. To fit into the
robot’s joint space into the three-dimensional spatial coordinates of
LfD algorithm settings, the data trajectories i.e., t 1 ; t 2 ; . . . ; t m consist of
the end effector A ¼ ðX; Y; ZÞ 2 R3 and enable DRL algorithm to
the environment observation si1 and robot motion action ai1 . The element
explore the feasible trajectory.
of s includes the task information, working environment, task conditions,
Reward (R) combines multiple safe motion planning tolerance
human information etc., and a is the robot movement demonstrated by
settings, including safety (e.g., human-robot distance 10 mm), and
the human-expert regarding the cases. Each demonstration of the dataset
task completion index evaluation (e.g., execution time  30 s), task
is represented as t i ¼ h si1 ; ai1 ; si2 ; ai2 ; . . . ; sinþ1 i and the whole dataset D
completion progress and success rate (e.g., target reaching deviation
is denoted as D ¼ fðS1 ; A1 Þ; ðS2 ; A2 Þ; ðS3 ; A3 Þ:::g. Essentially, the BC learn-
1 mm), which is denoted as R= (Rs,Rt).
ing is to approximate the maximum likelihood estimation of policy func-
Meanwhile, the actor-critic is employed to learn and control the
tion, which could minimise the difference between the model-generated
corresponding actions, which maximises the expected return JðuÞ and
state-action trajectory probability distribution (robot control policy) and
optimises the moving path in terms of safety:
the input trajectory probability distribution (human-expert policy):
" #
maxu Eðs;aÞ » D ½pu ðajsÞ ð1Þ XT
t
J ðuÞ ¼ Et » pu ðtÞ g rt ð2Þ
X t¼0
Q
s:t: b u ðajsÞ ¼ 1; 8 s 2 S
p where pu ðtÞ ¼ pðs0 Þ T
1 t¼0 ½ðstþ1 jst ; at Þpu ðat jst Þ is the probability distri-
a2A
bution over all possible state-action trajectories t ¼ ðs0 ; a0 ; s1 ; . . . ;
During the parameter optimisation process of the maximum like- aT
1 ; sT Þ; g t 2 ½0; 1 is the discount factor at time t, and du ðst Þ is the
lihood estimation, a policy pu ðsÞ is trained to minimise the difference state distribution under the policy up .
between robot behaviour patterns and human demonstration with
the objective function Eðs;aÞ » D k pu ðsÞ
a k 2 . In practice, pu ðsÞ 3. Case study
approximates the expert policy by using the deep neural network, To illustrate the performance of the HRC system when tackling
which is optimised via the gradient descent with an aim to gain the various uncertainties, comparative experiments are conducted on
optimal robot control policy function. some HRC assembly tasks in our lab environment, containing visual
4 P. Zheng et al. / CIRP Annals - Manufacturing Technology 72 (2023) 1
4

sensors (Azure Kinect and Intel D435), a GPU server (RTX 3080), one Table 3
human operator and collaborative robot (UR5). Firstly, the effective- Experimental results of the DRL-based safe motion planning.
ness of the proposed HDT model is evaluated with other baseline
Overlapped distance Success rate
methods for finding human abnormal behaviours and assisting No overlapping 98.7%
robots. Then, the LfD and DRL control policies are demonstrated and 0
15 cm (Blue) 91.7%
assessed in several human-assisting-robot and robot-assisting-human 15
30 cm (Red) 85.8%
uncertain tasks, while robot uncertainties are handled manually by
emergency stop and human inspection.
4. Conclusions and future work
3.1. Fine-grained HDT modelling for accurate robot assistance
The performance evaluation of the proposed HDT model mainly To ensure the successful implementation of HRC activities, this
consists of two parts: (1) human behaviour recognition accuracy and research proposed a systematic CI-based approach to handling the
(2) 3D human pose reconstruction error. For the prior one, RGB-D data human, task, and robot uncertainties integrally. The main scientific
was collected via an Azure Kinect camera capturing the HRC activities contributions of this work include: 1) the proposed dynamic graph
including 5 types of human-centric HRC subtasks: (1) dismantling, (2) embedding-based adaptive HRC task planning approach, 2) a novel
part picking, (3) robot handover, (4) robot guiding, and (5) robot stop- vision-based HDT modelling method for handling human uncertain-
ping. After trimming and cleaning the raw data, a total of 939 valid ties, and 3) the introduced LfD and DRL approaches for handling task/
action clips remained for evaluation purpose, of which 751 were used robot uncertainties and human ones, respectively. The performance
for training and 188 for testing. As for the evaluation of the fine- of the proposed CI has been reported in handling several HRC assem-
grained human pose reconstruction, we utilise MPJPE (Mean Per Joint bly task uncertainties with preliminary experimental results. In the
Position Error) metric to evaluate body and hand posture reconstruc- future, both multimodal intelligence- based HDT and advanced robot
tion errors, respectively. The comparative results are listed in Table 1, learning mechanisms can be explored to tackle multiple HRC scenar-
which clearly shows that our proposed HDT modelling scheme per- ios with uncertainties.
forms better on both behaviour recognition and human pose recon-
struction compared with previous approaches which only targeted at
a sole recognition task. The evaluation results demonstrated that the Declaration of Competing Interest
proposed HDT model is capable of conducting subsequent robot assis-
tance action planning based on human activity recognition. The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
Table 1 ence the work reported in this paper.
Comparative experimental results on the collected HRC data.

Method Body Hand Behaviour Example

pose pose recognition
error (mm) error (mm) accuracy Supplementary materials
Kanazawa et al. [13] 67.19
97.89%
Ours 52.14
98.94%
Supplementary material associated with this article can be found
in the online version at doi:10.1016/j.cirp.2023.04.057.
Hasson et al. [14]
50.95

Ours
38.41

References

[1] Wang L (2022) A Futuristic Perspective on Human-Centric Assembly. Journal of

3.2. LfD and DRL experiments for handling HRC uncertainties Manufacturing Systems 62:199–201.
[2] Wang L, Gao R, Vancza J, Kru € ger J, Wang XV, Makris S, Chryssolouris G (2019)
Human-assisting-robot, the LfD-based robot control experiments
Symbiotic Human-Robot Collaborative Assembly. CIRP Annals 68:701–726.
are conducted based on RoboMimic [15] dataset to evaluate the effec- [3] Li S, Zheng P, Liu S, Wang Z, Wang XV, Zheng L, Wang L (2023) Proactive Human-
tiveness of the Dagger integrated BC to handle HRC task uncertain-
Robot Collaboration: Mutual-Cognitive, Predictable, and Self-Organising Per-
ties, including picking, sorting, and assembly of bearing and base. spectives. Robotics and Computer-Integrated Manufacturing 81:1–30.
[4] Zheng P, Li S, Xia L, Wang L, Nassehi A (2022) A Visual Reasoning-Based Approach
Given 20 validated task trials, different numbers of expert demon- for Mutual-Cognitive Human-Robot Collaboration. CIRP Annals 71:377–380.
strations (i.e., 50, 100, 150, 180) are fed to Dagger, and their corre- [5] Fan J, Zheng P, Li S (2021) Vision-Based Holistic Scene Understanding Towards
sponding success rates are shown in Table 2. It is found that BC can Proactive Human
Robot Collaboration. Robotics and Computer-Integrated
Manufacturing 75:102304.
effectively extract human manipulation patterns to drive the robot [6] Meng K, Xu G, Peng X, Youcef-Toumi K, Li J (2022) Intelligent Disassembly of Elec-
for completing tasks. Meanwhile, the Dagger mechanism with BC tric-Vehicle Batteries: A Forward-Looking Overview. Resources, Conservation and
could significantly enhance the robustness of robot control policy Recycling 182:106207.
[7] James Wilson H, Daugherty PR (2018) Collaborative Intelligence: Humans and AI
with growing human demonstrations. are Joining Forces. Harvard Business Review 96(4):114–123.
Robot-assisting-human, the success rate of safe robotic motion [8] Li S, Zheng P, Fan J, Wang L (2021) Towards Proactive Human Robot Collaborative
planning is used to identify the effectiveness of tackling human Assembly: A Multimodal Transfer Learning-Enabled Action Prediction Approach.
IEEE Transactions on Industrial Electronics 69:8579–8588.
uncertainties. More specifically, it is evaluated based on the over- [9] Raatz A, Blankemeyer S, Recker T, Pischke D, Nyhuis P (2020) Task Scheduling
lapped distance between human operator and the work envelope of Method for HRC Workplaces Based on Capabilities and Execution Time Assump-
the robot. In this study, the Actor-Critic algorithm is designed and tions for Robots. CIRP Annals 69:13–16.
[10] Xiong Y, Zhang Y, Fu H, Wang W, Zhu Y, Yu PS (2019) DynGraphGAN: Dynamic
executed for robotic safe motion planning in the high fidelity Unity3D
Graph Embedding via Generative Adversarial Networks. Database Systems for
environment via 2 million steps. The resulting success rates of vari- Advanced Applications 24:536–552.
ous overlapped distances are shown in Table 3, which can largely [11] Bilberg A, Malik AA (2019) Digital Twin Driven Human
Robot Collaborative
improve safe motion planning for HRC works. Assembly. CIRP Annals 68(1):499–502.
[12] Liu S, Wang XV, Wang L (2022) Digital Twin-Enabled Advance Execution for
Human-Robot Collaborative Assembly. CIRP Annals 71(1):25–28.
[13] Kanazawa A, Black MJ, Jacobs DW, Malik J (2018) End-to-End Recovery of Human
Table 2 Shape and Pose. IEEE Conference on Computer Vision and Pattern Recognition,
Comparative experimental results on the Dagger-based 7122–7131.
LfD. [14] Hasson Y, Tekin B, Bogo F, Laptev I, Pollefeys M, Schmid C (2020) Leveraging Pho-
tometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruc-
Demonstrated tasks/trials 50 100 150 180
tion. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 571–580.
Success rate Picking 90% 96% 100% 100% [15] Mandlekar A., Xu D., Wong J., Nasiriany S., Wang C., Kulkarni R., Fei-Fei L., Savar-
Sorting 60% 82% 86% 88% ese S., Zhu Y., Martín-Martín R. (2021) What Matters in Learning From Offline
Assembly 50% 72% 82% 88%
Human Demonstrations for Robot Manipulation. arXiv preprint arXiv:2108.03298.