Skip to content

(5.x) Merge 4.x #23718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 327 commits into from
Jul 5, 2023
Merged

(5.x) Merge 4.x #23718

merged 327 commits into from
Jul 5, 2023

Conversation

asmorkalov
Copy link
Contributor

@asmorkalov asmorkalov commented May 31, 2023

Contrib PR: opencv/opencv_contrib#3501
Extra PR: opencv/opencv_extra#1066

#13879 from chacha21:REDUCE_SUM2
#20370 from ddacw:stub-gen-next
#21736 from anders-nylander:fix_android_sdk_tools_dependency
#22245 from rprasanth:4.x
#22275 from zihaomu:fp16_support_conv
#22441 from vrabaud:hls_while
#22471 from anna-khakimova:ak/fix_resize4lpi_tests
#22614 from zihaomu:add_std2DB_API
#22660 from zhouzq-thu:4.x
#22700 from kallaballa:capture_backened_debug_msg
#22750 from zihaomu:improve_blobFromImage
#22889 from D-Alex:patch-1
#23019 from tkram01:sampleIdxFix
#23020 from Wwupup:yunetv2
#23025 from savuor:backport3_stddev_calib_fix
#23047 from fengyuentau:layer_norm
#23055 from seanm:sprintf2
#23061 from WanliZhong:gemm_cuda
#23067 from sashashura:patch-3
#23073 from savuor:fix_graphcut_used_edges_dense
#23076 from inayd:22012-bugfixFillPoly
#23108 from crackwitz:issue-23107
#23111 from Yeatse:fix-apple-silicon-build
#23120 from alalek:fixup_22246_2
#23128 from Isydmr:update-fastNlMeansDenoising-documentation
#23131 from y-guyon:align_ptr_intrin_sse
#23138 from AleksandrPanov:aruco_fix_matchImagePoints
#23139 from AleksandrPanov:add_py_charuco_sample
#23148 from alalek:perf_drop_calibration
#23153 from stefan523:ChArUco_pre460_pattern_support
#23154 from tomoaki0705:handleNonBitExact
#23161 from dkurt:dnn_tflite
#23162 from zihaomu:issue_23151
#23165 from labeeb-7z:optimizeDistanceTransform
#23168 from genciberisha/bug/issue-22205_and_23105_encodeStructuredAppend_problem
#23172 from lilohuang:master
#23173 from tomoaki0705:fix_warning_master
#23185 from y-guyon:parallel_tsan_data_race
#23192 from zihaomu:clean_up_SIMD_code
#23196 from labeeb-7z:printOptionInRoiSelector
#23198 from mshabunin:fix-rvv-07
#23203 from asmorkalov:as/xcode_warning_supression
#23204 from whuaegeanse:fix-sift
#23209 from CSBVision:patch-3
#23210 from ibaiGorordo:rect_nfa_bugfix
#23211 from TolyaTalamanov:at/pipeline-modeling-tool-perf-alignment
#23217 from laitingsheng:4.x
#23219 from fengyuentau:add_gelu
#23222 from siilats:patch-1
#23224 from VadimLevin:dev/vlevin/cxx-named-arguments
#23225 from mshabunin:fix-clang-warnings
#23237 from hzcyf:feature/orbbec_femto_mega_support
#23243 from WanliZhong:accelerate_palm_det
#23246 from mshabunin:rvv07-support
#23247 from lilithgrigoryan:fix-focals-estimation
#23250 from tintou:./tintou/glib-req
#23255 from zihaomu:fused_cuda_naryeltwise
#23260 from tintou:tintou/gtk-reduce-diff
#23261 from vovka643:3.4
#23268 from VadimLevin:dev/vlevin/bindings-io-arg-modifiers-fix
#23271 from stefan523:aruco_testcase_fixes
#23274 from alalek:dnn_flatbuffers_builtin
#23275 from genciberisha:bug/issue-23249_detected_but_not_decoded_bug
#23276 from vovka643:flann_corrections
#23287 from alalek:dnn_tflite_fix_illegal_access
#23290 from vovka643:4.x_depricated_backends
#23292 from CSBVision:patch-4
#23296 from fengyuentau:fix_identifying_constant
#23300 from ippei-i:CAP_PROP_AUTO_WB-and-CAP_PROP_WHITE_BALANCE_BLUE_U_support_in_CAP_DSHOW
#23301 from shtern:fix_quaternion
#23305 from bhavitp:fix/calib3d/undistortion_grid
#23307 from alalek:simd_comparison_fix_misused_64f_guard
#23310 from hanliutong:fix_hal_compatibility
#23319 from fengyuentau:fix_zoo_issue_136
#23321 from a-sajjad72:patch-1
#23325 from dkurt:dnn_input_info
#23341 from TuNanTang:3.4
#23342 from n0099:#23335
#23343 from zihaomu:fix_test_onnx_conf
#23344 from anderskiaer:singlefilejs
#23350 from spikethehobbitmage:4.x
#23353 from tingboliao:4.x
#23354 from zihaomu:issue_23351
#23356 from TuNanTang:OptimizeFixFitEllipseSample
#23357 from zihaomu:fix_winograd_error_32bit
#23358 from Abdurrahheem:fix_doc_dnn_custom_layers
#23363 from vovka643:4.x_generate_charuco
#23367 from LaurentBerger:msmf_doc
#23371 from cudawarped:cuda_add_futher_python_interop
#23375 from mshabunin:fix-v4l-verify
#23377 from LaurentBerger:findsample
#23380 from Zero-nnkn:pose_doc
#23381 from ct2034:fix/typo
#23383 from mshabunin:rvv-scalable-gcc
#23386 from asmorkalov:issue23147
#23388 from simonlynen:patch-2
#23390 from just-gull:bugfix.21401.fix-macos-crash-when-keypress-does-nothing
#23394 from thewoz:Cocoa-Scroll-Wheel
#23399 from AleksandrPanov:aruco_fix_board
#23401 from fengyuentau:fix_cann_layer_support
#23402 from LaurentBerger:I23400
#23409 from dkurt:dnn_tflite_quant
#23410 from Kumataro:3.4-fix23397-step1
#23417 from souch:souch-fix-parallel-backend-example
#23418 from mshabunin:fix-rvv-07-old
#23419 from dkurt:onnx_fixes
#23433 from Kumataro:4.x-fix23416
#23436 from siilats:patch-2
#23442 from LaurentBerger:tuto_findFile
#23443 from eplankin:3.4
#23444 from opencv:AnnaPetrovicheva-add-funding-button
#23446 from tantei3:bmp_fix
#23447 from onuralpszr:gradle80_namespace
#23448 from dmatveev:dm/gapi_fix_standalone_47
#23449 from Zwyx:fix-mat-zeros-rows-cols
#23451 from Zero2key:4.x
#23460 from gottagofaster236:fix_obs_virtual_camera
#23468 from migdall:migdall-issue-add-is-easier-pyintro-markdown
#23469 from gottagofaster236:use_nv12_for_obs_camera
#23475 from Abdurrahheem:lstm_fix_initialization
#23476 from fengyuentau:add_note_for_yunet
#23477 from TolyaTalamanov:at/handle-multimeta-giebackend
#23479 from vovka643:4.x_charuco_calib
#23482 from zihaomu:onnx_opset13_split
#23484 from AleksandrPanov:fix_warnings_quirc
#23485 from zihaomu:add_onnx_where
#23486 from vovka643:4.x_calibration_with_aruco
#23487 from Gaotianhong:4.x
#23491 from fengyuentau:patch_for_segment_anything
#23495 from smeng9:4.x
#23499 from cabelo:yolov4-tiny
#23501 from Abdurrahheem:additional_lstm_tests
#23502 from seanm:sprintf3
#23503 from seanm:issue13729
#23506 from MvWouden:patch-1
#23511 from zihaomu:issue_23465
#23513 from komakai:fix_unrecognized_selector
#23514 from tappytaps:mac-catalyst-fix
#23515 from vovka643:4.x_aruco_stereo_calibration
#23518 from mshabunin:security-update
#23528 from WanliZhong:issue23278
#23529 from dmatveev:dm/gapi_onnx_rt_1.14.1
#23531 from kaingwade:orbbec_python_sample
#23535 from komakai:fix_ocl_device_isintel
#23538 from Avasam:patch-1
#23541 from cudawarped:fix_3474
#23545 from Abdurrahheem:tests_lstm_init_no_hidden_states
#23555 from LaurentBerger:doc_format
#23557 from WanliZhong:eltwise_cpu_bug
#23560 from WanliZhong:eltwise_cuda_bug
#23565 from MaximSmolskiy:fix-function-name-in-comment
#23566 from seanm:atomic-bool
#23567 from seanm:UBSan-overflow
#23572 from gabrielnhn:depthmap_tutorial
#23575 from vovka643:4.x_aruco_calib3d_calibration
#23583 from siilats:patch-3
#23587 from vovka643:4.x_aruco_calibrate_py
#23594 from fdivitto:fdivitto-traincascade-patch
#23597 from dmatveev:dm/gapi_onnx_py_integration
#23598 from komakai:fix-objc-gen-baseclass
#23600 from dkurt:flatbuffers_23.5.9
#23601 from cudawarped:videocapture_threading
#23604 from dkurt:dnn_no_protobuf
#23606 from mshabunin:fix-ffmpeg-packet-limit
#23612 from cpoerschke:3.4-issue-21532
#23614 from Abdurrahheem:lstm_layout_attribute
#23618 from LaurentBerger:unused_var
#23619 from TinyTinni:pixel-info-font-color
#23625 from zihaomu:improve_conv
#23629 from asmorkalov:as/cpu_features_bindings
#23631 from asmorkalov:as/eigen_NOMINMAX_warning_fix
#23632 from asmorkalov:as/AffineFeature_mask_fix_3.4
#23634 from dkurt:fix_nearest_exact
#23638 from soyszala:issue#23633
#23641 from zihaomu:ci_of_vulkan
#23645 from Abdurrahheem:ash/tf_init_input_check
#23646 from dkurt:dnn_ie_region_fix
#23652 from rahulnoronha:patch-1
#23655 from fengyuentau:qlinearsoftmax
#23656 from peters:patch-2
#23657 from asmorkalov:as/uyvy_cvtColor_size_check
#23662 from akormous:docfix
#23671 from zihaomu:fix_potential_bug
#23672 from mshabunin:fix-javadoc17
#23674 from dkurt:py_cv_maketype
#23679 from dkurt:py_cv_type_macro
#23682 from asmorkalov:as/openjpeg_2.5.0
#23684 from LaurentBerger:I23683
#23686 from LaurentBerger:I23685
#23690 from chacha21:rotatedRectangleIntersection_precision
#23691 from dkurt:pycv_float16_fixes
#23694 from dkurt:update_matchTemplateMask
#23695 from triple-Mu:triplemu/fix-python-binding
#23698 from cpoerschke:4.x-pr-21959-perf
#23703 from Abdurrahheem:ash/graph_simplifier
#23707 from Orange-OpenSource:ONNX_Squeeze_negative_index_for_the_axes_attribute
#23638 from soyszala:issue#23633
#23214 from asmorkalov:as/xcode_warning_supression3.4
#23349 form zihaomu:optimize_vulkan_dnn
#22245 from rprasanth:4.x
#23694 from dkurt:update_matchTemplateMask
#23209 from CSBVision:patch-3
#23124 from vicsyl:feature/dog_precision
#23270 from vpisarev:sift_double_old_default
#23272 from vpisarev:sift_double_old_default_upd
#23613 from fengyuentau:refactor_reduce

Previous "Merge 4.x": #23189

asmorkalov and others added 30 commits March 15, 2023 13:22
Optimize&Fix fitEllipse sample
…layers

Minor grammatical fixes to dnn_custom_layers doc file
Added argument to print notice in `roiSelector.cpp`

Related Issue : opencv#23175

I've added a printNotice argument to `selectROI` (and it's overload) and `selectROIs` functions.
I've also updated the function declarations in `highgui.hpp`.
Tested by building locally.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added regression test for the flipped images
…_PROP_WHITE_BALANCE_BLUE_U_support_in_CAP_DSHOW

Support VideoCapture CAP_PROP_AUTO_WB and CV_CAP_PROP_WHITE_BALANCE_BLUE_U for DShow

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [OK] I agree to contribute to the project under Apache 2 License.
- [OK] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [OK] The PR is proposed to the proper branch
- [OK] There is a reference to the original bug report and related work
opencv#19621
opencv#21408

### Before apply this pull request console output.

before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1

after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1

after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1

after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: -1

### After apply this pull request console output.

before AWB setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0

after AWB disable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 0

after AWB enable setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 4000
CAP_PROP_AUTO_WB: 1

after Manual WB(and Disable AWB) setting
CAP_PROP_WHITE_BALANCE_BLUE_U: 2000
CAP_PROP_AUTO_WB: 0

### Test Code
[OpenCvVideoCapTest.zip](https://github.com/opencv/opencv/files/10825399/OpenCvVideoCapTest.zip)
Propagate inputs info for ONNX and TFLite models

### Pull Request Readiness Checklist

Needed for generic applications such as benchmarking pipelines. So OpenCV can tell about the default input shapes specified in the models.

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add possibility for disabling inlining `wasm` in `opencv.js`
…ected_but_not_decoded_bug

Added QR_Code data flip support, flip and retry after first ECC failure
highgui: Set hard GLib requirement to >=2.32
Fix error of `POSE_PAIRS` in pose estimation doc
…ing keypoints prior to region growing

This makes LineSegmentDetector deterministic by using stable_sort for ordering points by norm. Without this change the region growing in LSD is non-determinstic and thus the returned lines are changing between invocations.

This is a replacement for opencv#23370
Fix for using sampleIdx to limit training data
Make LineSegmentDetector deterministic by using stable_sort
asmorkalov and others added 3 commits May 30, 2023 16:33
…egative_index_for_the_axes_attribute

Addition of normalize_axis to ONNXImporter::parseSqueeze
…ction_precision

better accuracy for _rotatedRectangleIntersection() (proposal for opencv#23546) opencv#23690

_rotatedRectangleIntersection() can be (statically) customized to use double instead of float for better accuracy
this is a proposal for experimentation around opencv#23546

for better accuracy, _rotatedRectangleIntersection() could use double. It will still return cv::Point2f list for backward compatibility, but the inner computations are controlled by a typedef

- [X] I agree to contribute to the project under Apache 2 License.
- [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [X] The PR is proposed to the proper branch
- [X] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
…tion

G-API: Integration branch for ONNX & Python-related changes opencv#23597

# Changes overview

## 1. Expose ONNX backend's Normalization and Mean-value parameters in Python

* Since Python G-API bindings rely on `Generic` infer to express Inference, the `Generic` specialization of `onnx::Params` was extended with new methods to control normalization (`/255`) and mean-value; these methods were exposed in the Python bindings
* Found some questionable parts in the existing API which I'd like to review/discuss (see comments)

UPD:
1. Thanks to @TolyaTalamanov normalization inconsistencies have been identified with `squeezenet1.0-9` ONNX model itself; tests using these model were updated to DISABLE normalization and NOT using mean/value.
2. Questionable parts were removed and tests still pass.

### Details (taken from @TolyaTalamanov's comment):

`squeezenet1.0.*onnx` - doesn't require scaling to [0,1] and mean/std because the weights of the first convolution already scaled. ONNX documentation is broken. So the correct approach to use this models is:

1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44 but without normalization step:
```
# DON'T DO IT:
# mean_vec = np.array([0.485, 0.456, 0.406])
# stddev_vec = np.array([0.229, 0.224, 0.225])
# norm_img_data = np.zeros(img_data.shape).astype('float32')
# for i in range(img_data.shape[0]):
#     norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
#     # add batch channel
#     norm_img_data = norm_img_data.reshape(1, 3, 224, 224).astype('float32')
#     return norm_img_data

# INSTEAD
return img_data.reshape(1, 3, 224, 224)
```

2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', False)
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.

---

`squeezenet1.1.*onnx` - requires scaling to [0,1] and mean/std - onnx documentation is correct.
1. ONNX: apply preprocessing from the documentation: https://github.com/onnx/models/blob/main/vision/classification/imagenet_preprocess.py#L8-L44
2. G-API: Convert image from BGR to RGB and then pass to `apply` as-is with configuring parameters:
```
net = cv.gapi.onnx.params('squeezenet', model_filename)
net.cfgNormalize('data_0', True) // default
net.cfgMeanStd('data_0', [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
```
**Note**: Results might be difference because `G-API` doesn't apply central crop but just do resize to model resolution.

## 2. Expose Fluid & kernel package-related functionality in Python

* `cv::gapi::combine()`
* `cv::GKernelPackage::size()` (mainly for testing purposes)
* `cv::gapi::imgproc::fluid::kernels()`

Added a test for the above.

## 3. Fixed issues with Python stateful kernel handling

Fixed error message when `outMeta()` of custom python operation fails.

## 4. Fixed various issues in Python tests

1. `test_gapi_streaming.py` - fixed behavior of Desync test to avoid sporadic issues
2. `test_gapi_infer_onnx.py` - fixed model lookup (it was still using the ONNX Zoo layout but was NOT using the proper env var we use to point to one).

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Copy link
Contributor

@opencv-alalek opencv-alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Looks like there is missing port of #23305 (new file is modules/3d/src/calibration_base.cpp)
  2. Missing GHA changes: +Vulkan
  3. Description: PRs like #16189 are already merged (by previous "Merge 4.x" #23189)
  4. Description: need to properly handle missing "direct" merges (with malformed merge commit messages) like:
+ 5025f29378 speed up vulkan dnn, and support ios and apple m1 chip. (#23349)
+ 923dbcc58f different interpolation by double image (#23124)
+ ca48e217f1 fixed another SIFT constructor (#23272)
- eefee8574a dnn: refactor reduce (#23613)
- f48939c2d7 temporarily set "enable_precise_upscale=false" by default to avoid sporadic failures in regression tests (#23270)

@asmorkalov asmorkalov mentioned this pull request Jun 27, 2023
6 tasks
@asmorkalov
Copy link
Contributor Author

#23305 (new file is modules/3d/src/calibration_base.cpp) ported in #23680 and should be merged first.

@asmorkalov
Copy link
Contributor Author

#23680 closed. All required changes included into this pr.

@asmorkalov asmorkalov force-pushed the 5.x-merge-4.x branch 2 times, most recently from b83f953 to da0865b Compare July 4, 2023 16:47
@opencv-alalek
Copy link
Contributor

asmorkalov wants to merge 327 commits into opencv:5.x from asmorkalov:5.x-merge-4.x

note: 5.x-merge-4.x

but commit message is:

Merge branch '4.x' into 5.x-merge-4.x-origin

note: 5.x-merge-4.x-origin

How did you get -origin here?

According to maintainer's guidelines -origin branch is not used for preparing the merge patch.
That branch is used for tracking history only. That branch is updated after the PR merge. You should not touch that branch before that.

Please follow the maintainer's guidelines.

Copy link
Contributor

@opencv-alalek opencv-alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@asmorkalov asmorkalov merged commit 5af40a0 into opencv:5.x Jul 5, 2023
asmorkalov added a commit to asmorkalov/ci-gha-workflow that referenced this pull request Jul 6, 2023
Related PR in main repo: opencv/opencv#23718

- Removed barcode module tests as it was migrated to objdetect
- Removed QR code test blacklist
asmorkalov added a commit to opencv/ci-gha-workflow that referenced this pull request Jul 6, 2023
Related PR in main repo: opencv/opencv#23718

- Removed barcode module tests as it was migrated to objdetect
- Removed QR code test blacklist
@asmorkalov asmorkalov mentioned this pull request Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.