Skip to content

dnn(test): re-enable tests which works with OpenVINO 2021.4.x #21167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 4, 2021
Merged

dnn(test): re-enable tests which works with OpenVINO 2021.4.x #21167

merged 1 commit into from
Dec 4, 2021

Conversation

alalek
Copy link
Member

@alalek alalek commented Dec 1, 2021 •
edited
Loading

3.4 part: #21173

Before (2021-12-01):

[ SKIPSTAT ] 1338 tests skipped
[ SKIPSTAT ] TAG='mem_6gb' skip 7 tests
[ SKIPSTAT ] TAG='verylong' skip 7 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_myriadx' skip 193 tests
[ SKIPSTAT ] TAG='dnn_skip_ocl' skip 4 tests (1 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ocl_fp16' skip 36 tests (3 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie' skip 0 tests (343 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_ngraph' skip 355 tests (189 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_cpu' skip 1 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl' skip 21 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl_fp16' skip 71 tests (2 times in extra skip list)
[ SKIPSTAT ] TAG='skip_other' skip 643 tests
[==========] 8470 tests from 85 test cases ran.

After (2021-12-01):

[ SKIPSTAT ] 1008 tests skipped
[ SKIPSTAT ] TAG='mem_6gb' skip 7 tests
[ SKIPSTAT ] TAG='verylong' skip 7 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_myriadx' skip 170 tests
[ SKIPSTAT ] TAG='dnn_skip_ocl' skip 4 tests (1 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ocl_fp16' skip 36 tests (3 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie' skip 0 tests (214 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_ngraph' skip 74 tests (218 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_cpu' skip 3 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl' skip 31 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl_fp16' skip 33 tests (2 times in extra skip list)
[ SKIPSTAT ] TAG='skip_other' skip 643 tests
[==========] 8498 tests from 85 test cases ran.

After (2021-12-03) and merge #21173:

[ SKIPSTAT ] 1018 tests skipped
[ SKIPSTAT ] TAG='mem_6gb' skip 7 tests
[ SKIPSTAT ] TAG='verylong' skip 7 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_myriadx' skip 174 tests
[ SKIPSTAT ] TAG='dnn_skip_ocl' skip 4 tests (1 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ocl_fp16' skip 36 tests (3 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie' skip 0 tests (218 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_ngraph' skip 74 tests (222 times in extra skip list)
[ SKIPSTAT ] TAG='dnn_skip_ie_cpu' skip 3 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl' skip 31 tests
[ SKIPSTAT ] TAG='dnn_skip_ie_ocl_fp16' skip 33 tests (2 times in extra skip list)
[ SKIPSTAT ] TAG='skip_other' skip 649 tests
[==========] 8519 tests from 85 test cases ran.
List of re-enabled tests 
DNNTestNetwork.MobileNet_SSD_v1_TensorFlow_Different_Width_Height/1
Layer_Test_Halide/FullyConnected.Accuracy*
Layer_Test_Halide/LRN.Accuracy*
Test_Darknet_layers.shortcut/1
Test_Darknet_nets_async.Accuracy*
Test_ONNX_layers.AvePool1d/1
Test_ONNX_layers.PoolConv1d/1
Test_ONNX_layers.MaxPoolSigmoid1d/1
Test_ONNX_layers.MaxPool1d_Twise/1 
Test_ONNX_nets.Inception_v1/1
Test_TensorFlow_layers.ave_pool_same/1
Test_TensorFlow_layers.concat_3d/1
Test_TensorFlow_layers.padding/1
Test_Torch_layers.net_lp_pooling_power/1

Async.create_layer_pipeline_set_and_forward_all/0
Async.create_layer_pipeline_set_and_forward_all/1
Async.create_layer_pipeline_set_and_forward_all/2
Async.create_layer_pipeline_set_and_forward_all/3
Layer_Test_Halide/Eltwise.Accuracy*  (2 OCL tests with bad accuracy)
Test_Caffe_layers.InnerProduct/0
Test_Caffe_layers.InnerProduct/1
Test_Caffe_nets.Axpy/{0,1,2,3}
Test_Caffe_nets.FasterRCNN_vgg16/0
Test_Darknet_layers.region/1
Test_ONNX_layers.AvePooling3D/{1,2,3}
Test_ONNX_layers.Compare/0
Test_ONNX_layers.CompareSameDims/0
Test_ONNX_layers.Elementwise_{Ceil,Floor,Log,Round,Sqrt,not}/*
Test_ONNX_layers.Eltwise3D/*
Test_ONNX_layers.AvePooling3D/*
Test_ONNX_layers.PoolConv3D/*
Test_ONNX_layers.MaxPooling3D/0
Test_TensorFlow_layers.MaxPooling3D/0
Test_TensorFlow_layers.AvePooling3D/*
Test_ONNX_layers.ReduceMean3D/1
Test_ONNX_nets.Resnet34_kinetics/{2,3}
Test_TensorFlow_layers.Convolution3D/*
Test_TensorFlow_layers.lstm/0
Test_TensorFlow_layers.reshape/*
Test_TensorFlow_layers.resize_nearest_neighbor_half_pixel/*
Test_TensorFlow_layers.slim_batch_norm/*
Test_TensorFlow_layers.tf2_prelu/1
Test_Torch_layers.net_lp_pooling_power/1
Test_Torch_layers.net_conv_gemm_lrn/1

SKIP:
DNNTestOpenVINO.models/5
DNNTestOpenVINO.models/8
Test_TensorFlow_layers.lstm/1
Test_Darknet_nets.YOLOv4/3
Test_Darknet_nets.YOLOv4_tiny/1
Test_Darknet_nets.YOLOv4_tiny/3

force_builders=Custom,Custom Win,Custom Mac

build_image:Custom=ubuntu-openvino-2021.4.1:20.04
Xbuild_image:Custom=ubuntu-openvino-2021.3.0:20.04
Xbuild_image:Custom=ubuntu-openvino-2021.2.0:20.04
Xbuild_image:Custom=ubuntu-openvino-2020.3.0:16.04
build_image:Custom Win=openvino-2021.4.2
Xbuild_image:Custom Win=openvino-2021.3.0-vc16
build_image:Custom Mac=openvino-2021.4.0

test_modules:Custom=dnn,python2,python3,java
test_modules:Custom Win=dnn,python2,python3,java
test_modules:Custom Mac=dnn,python2,python3,java

buildworker:Custom=linux-1
test_opencl:Custom=ON
test_bigdata:Custom=1
test_filter:Custom=*

test_bigdata:Custom Win=1
buildworker:Custom Win=windows-3
test_opencl:Custom Win=ON
test_filter:Custom Win=*

@alalek alalek mentioned this pull request Dec 2, 2021
Merged
@alalek
Copy link
Member Author

alalek commented Dec 4, 2021

👍

@opencv-pushbot opencv-pushbot merged commit 73318fd into opencv:4.x Dec 4, 2021
@alalek alalek mentioned this pull request Dec 30, 2021
Merged
@alalek alalek mentioned this pull request Feb 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@alalek alalek

Labels
category: dnn test
Projects
None yet
Milestone
4.5.5
Development

Successfully merging this pull request may close these issues.
2 participants
@alalek @opencv-pushbot