Skip to content

Add Robot-World/Hand-Eye calibration function #18203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 27, 2020
Merged

Add Robot-World/Hand-Eye calibration function #18203

merged 1 commit into from
Aug 27, 2020

Conversation

catree
Copy link
Contributor

@catree catree commented Aug 27, 2020 •
edited by alalek
Loading

Merge with extra: opencv/opencv_extra#803

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

Code ported from Mili Shah's Matlab code with the following mention:

Copyright [2014] [Mili Shah]
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

The two implemented methods are:

  • M. Shah, "Solving the robot-world/hand-eye calibration problem using the kronecker product", Journal of Mechanisms and Robotics, vol. 5, p. 031007, 2013.
  • A. Li, L. Wang, and D. Wu, "Simultaneous robot-world and hand-eye calibration using dual-quaternions and kronecker product", International Journal of Physical Sciences, vol. 5, pp. 1530–1536, 2010.

These two methods seems to perform quite well according to Solving the Robot-World Hand-Eye(s) Calibration Problem with Iterative Methods. Moreover they are easy to implement thanks to the Matlab code.
Non-linear iterative methods are most likely more accurate, but most available implementation require the Ceres solver and I did not found an easy and clean implementation.

@catree catree mentioned this pull request Aug 27, 2020
Merged
alalek
alalek approved these changes Aug 27, 2020
View reviewed changes
Copy link
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thank you for contribution 👍

@opencv-pushbot opencv-pushbot merged commit 192e5c8 into opencv:master Aug 27, 2020
@zarathustr
Copy link

I wonder if there is any need to introduce globally optimal hand-eye calibration solver to OpenCV? Existing solvers in OpenCV are analytical and cannot guarantee global optimality, in the case of degeneration or missing measurements. We have just released one library https://github.com/zarathustr/LibQPEP, which was just published in IEEE Transactions on Robotics. This library only requires Eigen for computation so is light weight. Maybe we can make an OpenCV-based version and merge that?

@catree
Copy link
Contributor Author

catree commented Aug 29, 2022

@zarathustr

As a simple OpenCV user, I would definitively support the introduction of state of the art algorithms for PnP functions and Hand-Eye calibration functions.

Hand-Eye calibration is needed when mixing computer vision with robotics, e.g. like visual-servoing tasks.
I have added these functions since there are easy to be ported in C++, but as you have already noticed are analytical and can be easily prone to measurement errors.

I believe Eigen requirement can be ok as long as it is documented and appropriately handled to warn the user if OpenCV has not been compiled with Eigen support.
For sure, a pure C++ porting would be ideal.

About PnP functionality, I think solvePnPGeneric() (in addition to solvePnP() function) can be adapted if you want to return also covariance or if an approach can return multiple solutions.

About Point-to-plane Registration, it fits the OE 33. 3D Module I think.

OpenCV is licensed under the Apache-2.0 license, and thus your code should also comply with it.

@catree
Copy link
Contributor Author

catree commented Aug 29, 2022 •
edited
Loading

There is also #15650 work for a better, class-based, interface of the PnP functions.

I believe moving slowly is the way to go, but in the long term the class-based approach should be adopted.

What about adding Eigen as a private third-party library into OpenCV?
Eigen is pretty standard for Maths, this should simplify the integration of advanced algorithms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alalek alalek alalek approved these changes

Assignees

@alalek alalek

Labels
category: calib3d feature
Projects
None yet
Milestone
4.5.0
Development

Successfully merging this pull request may close these issues.
5 participants
@catree @zarathustr @alalek @Miguel-OYeah @opencv-pushbot