G-API: Add support to set workload type dynamically in both OpenVINO and ONNX OVEP #27460

fcmiron · 2025-06-19T10:41:52Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

asmorkalov · 2025-06-25T07:13:16Z

@AsyaPronina Friendly reminder.

AsyaPronina · 2025-06-27T17:01:57Z

Hello Dear @fcmiron, could I please ask you to rename this PR to take into account that this change is only for OpenVINO backend?

modules/gapi/include/opencv2/gapi/infer/ov.hpp

modules/gapi/src/backends/ov/govbackend.cpp

modules/gapi/include/opencv2/gapi/infer/ov.hpp

fcmiron · 2025-07-02T13:23:54Z

@asmorkalov can you retrigger the default check?

silviuhrehoretintel · 2025-07-03T12:42:08Z

rebuild default

modules/gapi/include/opencv2/gapi/infer/ov.hpp

modules/gapi/src/backends/onnx/gonnxbackend.cpp

modules/gapi/src/backends/ov/govbackend.cpp

AsyaPronina · 2025-07-14T15:28:23Z

modules/gapi/src/backends/ov/govbackend.cpp

+    auto workload_arg = cv::gapi::getCompileArg<cv::gapi::wip::ov::WorkloadTypeRef>(compileArgs);
+    if(workload_arg.has_value()) {
+        m_workload = workload_arg;
+        m_workloadId = m_workload.value().get().addListener(std::bind(&GOVExecutable::setWorkLoadType, this, std::placeholders::_1));


I don't think it is a workloadId, actually it is a listenerId for the workload_type struct. By the way, will we have more than one listener to this struct?

In protopipe you can have multiple streams/graphs structures so if you pass the same workload type object to compile all of them, to update the workload type for all of them to the same value, then there will be more than one listener

Do we need some mechanism to differ two separate WorkloadType variables for two different OV Infer nodes created in Protopipe? Could you re-iterate with the feature requester on it?

They don't need to differentiate between two different OV Infer nodes, they only need to update the workload type for the entire stream

modules/gapi/src/backends/ov/govbackend.cpp

modules/gapi/src/backends/ov/govbackend.hpp

modules/gapi/src/backends/ov/govbackend.cpp

AsyaPronina · 2025-07-17T11:55:11Z

modules/gapi/include/opencv2/gapi/gcommon.hpp

+    std::unordered_map<unsigned int, Callback> listeners;
+    unsigned int next_id = 0;
+ public:
+    unsigned int addListener(Callback cb) {


I think you can pass by reference here

AsyaPronina · 2025-07-17T11:56:09Z

modules/gapi/include/opencv2/gapi/gcommon.hpp

@@ -291,6 +292,26 @@ namespace detail
    };
 }

+class WorkloadType {
+    using Callback = std::function<void(const std::string &type)>;
+    std::unordered_map<unsigned int, Callback> listeners;


Better to place private variables at the end of the class with explicit usage of private access modifier just for consistency with other code

I am still not sure about Listener representation here. Can we somehow hide its callable and identifier in some kind of separate class? Then, we can use just std::unordered_set in here. I guess, each object of that Listener class can have a hash calculated basing on the callback function pointer. So, Listener objects don't need actual id fields, as their hashes will be their identifiers. What do you think?

This way we won't store m_workloadType along with m_workloadListenerId in backends, where m_workloadListenerId will have valid value only after adding a callback to the listeners of the m_workloadType. We will store m_workloadType along with m_workloadListener instead, and this m_workloadListener will be independent from m_workloadType algorithms.

AsyaPronina · 2025-07-17T11:57:38Z

modules/gapi/include/opencv2/gapi/gcommon.hpp

+ public:
+    unsigned int addListener(Callback cb) {
+        unsigned int id = next_id++;
+        listeners.emplace(id, std::move(cb));


Can we use next_id++ right here?

AsyaPronina · 2025-07-17T11:58:35Z

modules/gapi/include/opencv2/gapi/infer/onnx.hpp

 } // namespace onnx
 } // namespace gapi
+namespace detail {
+template<> struct CompileArgTag<std::shared_ptr<cv::gapi::onnx::WorkloadTypeONNX>> {


I think you can safely use WorkloadTypeOnnxPtr here

AsyaPronina · 2025-07-17T11:59:22Z

modules/gapi/include/opencv2/gapi/infer/onnx.hpp

@@ -752,8 +752,16 @@ class Params<cv::gapi::Generic> {
    std::string m_tag;
 };

+class WorkloadTypeONNX : public WorkloadType {};
+using WorkloadTypeOnnxPtr = std::shared_ptr<cv::gapi::onnx::WorkloadTypeONNX>;


If we preserve this using, suggest to have ONNX with capitals here to be aligned with the type name above

AsyaPronina · 2025-07-17T15:33:13Z

modules/gapi/src/backends/ov/govbackend.cpp

+    auto workload_arg = cv::gapi::getCompileArg<cv::gapi::wip::ov::WorkloadTypeRef>(compileArgs);
+    if(workload_arg.has_value()) {
+        m_workload = workload_arg;
+        m_workloadId = m_workload.value().get().addListener(std::bind(&GOVExecutable::setWorkLoadType, this, std::placeholders::_1));


Do we need some mechanism to differ two separate WorkloadType variables for two different OV Infer nodes created in Protopipe? Could you re-iterate with the feature requester on it?

AsyaPronina · 2025-07-17T16:13:54Z

modules/gapi/include/opencv2/gapi/gcommon.hpp

@@ -291,6 +292,26 @@ namespace detail
    };
 }

+class WorkloadType {


I don't think that gcommon.hpp is a right place to put WorkloadType in. This file is far more general for such kind of compile arguments, as graph_dump_path and use_threaded_executor, mentioned here are actually backend-independent.

Might be some additional file can be placed here: https://github.com/opencv/opencv/tree/4.x/modules/gapi/include/opencv2/gapi/infer. Something like workload_type.hpp?

AsyaPronina · 2025-07-17T16:19:14Z

modules/gapi/src/backends/onnx/gonnxbackend.cpp

 public:
    explicit ONNXCompiled(const gapi::onnx::detail::ParamDesc &pp);
+    ~ONNXCompiled();
+    void configureWorkloadType(cv::gapi::onnx::WorkloadTypeOnnxPtr workload);


Might be we can rename it to listenToWorkloadType(), if it will not change ONNX workload type state after all the changes.

AsyaPronina · 2025-07-17T16:26:21Z

modules/gapi/src/backends/ov/govbackend.cpp

+        compiled.compiled_model.set_property({{"WORKLOAD_TYPE", ::ov::WorkloadType::EFFICIENT}});
+    }
+    else {
+        GAPI_LOG_WARNING(NULL, "Unknown value for WORKLOAD_TYPE");


Should this be warning or an exception?

I think warning is enough because if the value is not valid, the workload type will not be updated but it will still work

AsyaPronina · 2025-07-17T16:26:48Z

modules/gapi/src/backends/ov/govbackend.cpp

@@ -1541,7 +1541,13 @@ cv::gimpl::ov::GOVExecutable::GOVExecutable(const ade::Graph &g,
                                            const cv::GCompileArgs &compileArgs,
                                            const std::vector<ade::NodeHandle> &nodes)
    : m_g(g), m_gm(m_g) {
-
+#if defined HAVE_INF_ENGINE && INF_ENGINE_RELEASE >= 2024030000


As we have HAVE_INF_ENGINE guard already, I suggest to use only the second one

fh9621063 · 2025-07-25T17:57:33Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.

To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV

The PR is proposed to the proper branch

There is a reference to the original bug report and related work

There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.

The feature is well documented and sample code can be built with the project CMake

ok

asmorkalov · 2025-08-11T10:21:01Z

@AsyaPronina What is the PR status?

add support to set workloadtype dynamically

8b7c27e

fcmiron changed the title ~~Add support to set workloadtype dynamically~~ G-API: Add support to set workloadtype dynamically Jun 19, 2025

fcmiron marked this pull request as ready for review June 19, 2025 12:33

asmorkalov added the category: g-api / gapi label Jun 19, 2025

asmorkalov requested review from dmatveev and TolyaTalamanov June 19, 2025 13:03

asmorkalov added this to the 4.12.0 milestone Jun 19, 2025

fix build errors by converting to c++11 standard

0657025

dmatveev requested a review from AsyaPronina June 23, 2025 11:20

change workloadtype key and value

3b251b9

asmorkalov modified the milestones: 4.12.0, 4.13.0 Jun 26, 2025

AsyaPronina requested changes Jun 27, 2025

View reviewed changes

fcmiron changed the title ~~G-API: Add support to set workloadtype dynamically~~ G-API: Add support to set workloadtype in OpenVINO dynamically Jun 30, 2025

resolve comments

98c024a

fcmiron requested a review from AsyaPronina June 30, 2025 09:43

fcmiron added 3 commits June 30, 2025 14:14

fix build

d9caa27

add ifdefs to check openVINO release version

85c1989

fix ifdef

ef7fbe4

set workloadtype implementation for onnx ovep

183a3b9

fcmiron changed the title ~~G-API: Add support to set workloadtype in OpenVINO dynamically~~ G-API: Add support to set workload type dynamically in both OpenVINO and ONNX OVEP Jul 7, 2025

cleanup

aa1934a

asmorkalov assigned AsyaPronina Jul 14, 2025

AsyaPronina reviewed Jul 14, 2025

View reviewed changes

resolve comments

904b530

fcmiron requested a review from AsyaPronina July 15, 2025 13:56

fcmiron added 4 commits July 15, 2025 17:06

fix build

f87ab18

have separate classes for ONNX and OV workloadtype

1341b6b

fix build

c00b5ae

fix build

1d2ce9a

AsyaPronina reviewed Jul 17, 2025

View reviewed changes

fcmiron added 2 commits July 21, 2025 15:55

resolve comments

ec984dd

remove prints

c410763

fcmiron requested a review from AsyaPronina July 21, 2025 13:03

fcmiron added 2 commits July 21, 2025 16:09

remove whitespaces

52b3872

fix warnings

56ac9f2

Uh oh!

G-API: Add support to set workload type dynamically in both OpenVINO and ONNX OVEP #27460

Are you sure you want to change the base?

G-API: Add support to set workload type dynamically in both OpenVINO and ONNX OVEP #27460

Conversation

fcmiron commented Jun 19, 2025

Pull Request Readiness Checklist

Uh oh!

asmorkalov commented Jun 25, 2025

Uh oh!

AsyaPronina commented Jun 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fcmiron commented Jul 2, 2025

Uh oh!

silviuhrehoretintel commented Jul 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fcmiron Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fh9621063 commented Jul 25, 2025

Pull Request Readiness Checklist

Uh oh!

asmorkalov commented Aug 11, 2025

Uh oh!

Uh oh!

fcmiron Jul 15, 2025 •

edited

Loading