Skip to content

Added timestamps support for Orbbec SDK backend in VideoIO. #27610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 4, 2025

Conversation

asmorkalov
Copy link
Contributor

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov
Copy link
Contributor Author

@fengyuentau @jia-kai Could you take a look?

@fengyuentau
Copy link
Member

@asmorkalov Jia's GitHub account is @kaingwade .

@fengyuentau
Copy link
Member

@asmorkalov Which model can we use for testing this PR?

@asmorkalov
Copy link
Contributor Author

asmorkalov commented Jul 31, 2025

I use Astra 2, but I expect it should work for all cameras.

@kaingwade
Copy link
Contributor

Astra 2 needs a firmware update to work properly after #24877, but Orbbec hasn't provided the update, and there is no plan to do so as far as I know.
I tested this PR too on Ubuntu with Astra 2, still fails to output the depth image, just like before.
@asmorkalov

@asmorkalov
Copy link
Contributor Author

I replaced the resolution setup like this:

diff --git a/modules/videoio/src/cap_obsensor_liborbbec.cpp b/modules/videoio/src/cap_obsensor_liborbbec.cpp
index ac581d81dc..53bfeb2cb3 100644
--- a/modules/videoio/src/cap_obsensor_liborbbec.cpp
+++ b/modules/videoio/src/cap_obsensor_liborbbec.cpp
@@ -39,13 +39,15 @@ VideoCapture_obsensor::VideoCapture_obsensor(int)
     config = std::make_shared<ob::Config>();
     pipe = std::make_shared<ob::Pipeline>();
     auto colorProfiles = pipe->getStreamProfileList(OB_SENSOR_COLOR);
-    auto colorProfile = colorProfiles->getProfile(OB_PROFILE_DEFAULT);
+    // auto colorProfile = colorProfiles->getProfile(OB_PROFILE_DEFAULT);
+    auto colorProfile = colorProfiles->getVideoStreamProfile(1920, 1080, OB_FORMAT_MJPG, OB_FPS_ANY);
+
     config->enableStream(colorProfile->as<ob::VideoStreamProfile>());
 
     auto depthProfiles = pipe->getStreamProfileList(OB_SENSOR_DEPTH);
-    auto depthProfile = depthProfiles->getProfile(OB_PROFILE_DEFAULT);
+    // auto depthProfile = depthProfiles->getProfile(OB_PROFILE_DEFAULT);
+    auto depthProfile = depthProfiles->getVideoStreamProfile(1600, 1200, OB_FORMAT_Y14, 5); //OB_FPS_ANY);
     config->enableStream(depthProfile->as<ob::VideoStreamProfile>());
-
     config->setAlignMode(ALIGN_D2C_SW_MODE);
 
     pipe->start(config, [&](std::shared_ptr<ob::FrameSet> frameset) {

And my camera works well with Orbbec SDK. Could you try it too? In case if it works as expected, I propose to merge the patch. I have another one.

@asmorkalov asmorkalov force-pushed the as/orbbec_sdk_timestamps branch from 20a6df1 to 8496706 Compare August 1, 2025 13:04
@kaingwade
Copy link
Contributor

I use Astra 2, but I expect it should work for all cameras.

It works for Astra 2 on Ubuntu with OBSENSOR_USE_ORBBEC_SDK=ON. Tested Gemini 2 L and it also works.

I replaced the resolution setup like this:

diff --git a/modules/videoio/src/cap_obsensor_liborbbec.cpp b/modules/videoio/src/cap_obsensor_liborbbec.cpp
index ac581d81dc..53bfeb2cb3 100644
--- a/modules/videoio/src/cap_obsensor_liborbbec.cpp
+++ b/modules/videoio/src/cap_obsensor_liborbbec.cpp
@@ -39,13 +39,15 @@ VideoCapture_obsensor::VideoCapture_obsensor(int)
     config = std::make_shared<ob::Config>();
     pipe = std::make_shared<ob::Pipeline>();
     auto colorProfiles = pipe->getStreamProfileList(OB_SENSOR_COLOR);
-    auto colorProfile = colorProfiles->getProfile(OB_PROFILE_DEFAULT);
+    // auto colorProfile = colorProfiles->getProfile(OB_PROFILE_DEFAULT);
+    auto colorProfile = colorProfiles->getVideoStreamProfile(1920, 1080, OB_FORMAT_MJPG, OB_FPS_ANY);
+
     config->enableStream(colorProfile->as<ob::VideoStreamProfile>());
 
     auto depthProfiles = pipe->getStreamProfileList(OB_SENSOR_DEPTH);
-    auto depthProfile = depthProfiles->getProfile(OB_PROFILE_DEFAULT);
+    // auto depthProfile = depthProfiles->getProfile(OB_PROFILE_DEFAULT);
+    auto depthProfile = depthProfiles->getVideoStreamProfile(1600, 1200, OB_FORMAT_Y14, 5); //OB_FPS_ANY);
     config->enableStream(depthProfile->as<ob::VideoStreamProfile>());
-
     config->setAlignMode(ALIGN_D2C_SW_MODE);
 
     pipe->start(config, [&](std::shared_ptr<ob::FrameSet> frameset) {

Changed resolution works with Astra 2, but not with Gemini 2 L and Femto (possibly some other models as well):
[warning][1025160][ObException.cpp:5] Invalid input, No matched video stream profile found!

@asmorkalov
Copy link
Contributor Author

I set resolution and FPS profile according to the camera spec. Most probably so high depth resolution is not supported by other devices. Please approve the PR, I'll submit another PR with resolution setting.

@asmorkalov asmorkalov assigned asmorkalov and unassigned fengyuentau Aug 4, 2025
@asmorkalov asmorkalov merged commit 13846de into opencv:4.x Aug 4, 2025
28 checks passed
@asmorkalov asmorkalov mentioned this pull request Aug 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants