Or maybe solvePnPWithExtrinsicGuess but longer name.

There is still an issue about how the logic behind the function works in my opinion.
The following call with SOLVEPNP_EPNP flag:

solvePnPExtrinsicGuess(objectPoints, imagePoints, cameraMatrix, distCoeffs, rvec, tvec, SOLVEPNP_EPNP)

will not use the initial rvec and tvec but only SOLVEPNP_ITERATIVE flag will use it:

Indeed, EPnP algorithm does not need an initial solution to solve the PnP problem.
This is already confusing in the current method unfortunately.

solvePnPExtrinsicGuess without the SolvePnPType flags maybe?

SOLVEPNP_ITERATIVE flag is a combination of:

• opencv/modules/calib3d/src/calibration.cpp

  Line 1043 in b761ec0

  CV_IMPL void cvFindExtrinsicCameraParams2( const CvMat* objectPoints,

• with pose from Homography in the planar case

  opencv/modules/calib3d/src/calibration.cpp

  Line 1114 in b761ec0

  // a planar structure case (all M's lie in the same plane)

• with DLT method otherwise

  opencv/modules/calib3d/src/calibration.cpp

  Line 1170 in b761ec0

  // non-planar structure. Use DLT method

• and a Levenberg-Marquardt optimization if useExtrinsicGuess

  opencv/modules/calib3d/src/calibration.cpp

  Lines 1221 to 1223 in b761ec0

  	// refine extrinsic parameters using iterative algorithm
  	CvLevMarq solver( 6, count*2, cvTermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER,max_iter,FLT_EPSILON), true);
  	cvCopy( &_param, solver.param );

In my opinion, a better way would be to have (not in the current plan since this break how solvePnP works):

• a flag to use cvFindExtrinsicCameraParams2, but what name to use since there are two methods under the hood based on the planar / non-planar case
• a flag to only refine the solution and that would use the iterative algorithm with Levenberg-Marquardt optimization method
• solvePnPExtrinsicGuess() renamed to solvePnPRefine or something in this spirit, to have a method that just run the iterative non-linear optimization method to refined the pose?

@catree Thank you for looking on this!

I believe, longer name is not a problem with modern IDEs.

In theory there are several different implementations of iterative algorithms. OpenCV just have this one at this moment.

Possible new algorithms can be added via:

1. extending "flags" enumeration
2. adding new functions, like findChessboardCornersSB()

The second way is preferable if additionally we need to expose some algorithm parameters.
Also it is much harder to misuse parameters, like "iterative" SOLVEPNP_EPNP.

The first way sometimes produce monsters like this with method, param1, param2. Without documentation it is hard to say that they really mean.

According the second way, I see these alternative:

• solvePnPIterative() without flags as suggested.

There is similar problem with solvePnPRansac().
But solvePnPRansac() reuses solvePnP(), so the second way with adding of new functions for new methods of solvePnP() doesn't look well here.

What are the options?

• solvePnPIterative with a new flag to select the refinement method
• solvePnPIterative without flags, or solvePnPRefineLM

I like having one function with a flag to select the method.
But currently there is only implemented method (Levenberg-Marquardt). If we can be sure that all the optimization methods require the same parameters (something like max iterations and epsilon) a new flag could be introduced indeed.

Otherwise, we could play safe and introduce a new method for each new approach and try to minimize duplicate code internally?

For solvePnPRansac, the behavior is not clear/intuitive. If useExtrinsicGuess is set to true, currently only the final pose estimated with all the inliers should be done as an optimization problem if I recall well.

But if we have an initial guess, I would say it makes sense to use it also to determine the pose for the consensus step.

Also, why not use the pose that allows getting the biggest consensus set and just refine the pose with an iterative algorithm and all the inliers?

Well, this is the trade-off between api simplicity and exposure to method specific parameters. I agree it's too restrictive to assume refinement methods should have the same parameters. Furthermore, with the current api we have no exposure to LM refinement parameters, which is limiting. The most important parameters are stopping criteria and loss function (currently L2 but other robust losses are also used in the research literature to deal with outliers such as Huber or L1). I'm of course not suggesting different losses be implemented in this issue but it's worth knowing about.

Stepping back, the fundamental division between pnp methods is whether they are for locally improving an initial pose estimate (a PnpRefiner) or not (a PnpSolver). As you've said, it doesn't make sense to use a PnPSolver e.g. IPPE or EPnP with a "use extrinsicGuess" option. Similarly it doesn't make sense to use a PnPRefiner with useExtrinsicGuess as an option.

I would propose that Pnp methods be implemented as objects that inherit from two base classes (either PnPSolver or PnpRefiner). Method specific parameters would be passed at construction time.

We nevertheless need to avoid sacrificing simplicity for users who don't want to worry about specific solvers or custom parameters. This could be done with an encapsulator that constructs and executes default PnPSolver and PnPRefiner objects (chosen depending on whether the points are planar etc.) with default parameters.

@catree re "Also, why not use the pose that allows getting the biggest consensus set and just refine the pose with an iterative algorithm and all the inliers?"

The only problem with that is when pose estimation is ambiguous such as planar cases. The pose that provides the biggest consensus is not necessary the correct one in those situations.

@tobycollins I agree that using classes would provide more flexibility and cleaner interface, something similar to the Feature2D class.

I have proposed #14431 to add functions to refine a pose. This could be a first step toward something cleaner and useful.