diff --git a/.travis.yml b/.travis.yml index eaa76047e..e11a3ca3b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -46,7 +46,7 @@ env: #utPLSQL released version directory - UTPLSQL_DIR="utPLSQL_latest_release" - SELFTESTING_BRANCH=${TRAVIS_BRANCH} - - UTPLSQL_CLI_VERSION="3.1.7" + - UTPLSQL_CLI_VERSION="3.1.8" # Maven - MAVEN_HOME=/usr/local/maven - MAVEN_CFG=$HOME/.m2 diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index b034e81db..35ed7d778 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1225,11 +1225,11 @@ Finished in .035261 seconds ### Tags -Tag is a label attached to the test or a suite path. It is used for identification and execution a group of tests / suites that share same tag. +Tag is a label attached to the test or a suite. It is used for identification and execution of a group of tests / suites that share the same tag. -It allows us to group a tests / suites using a various categorization and place a test / suite in multiple buckets. Same tests can be group with other tests based on the functionality , frequency, type of output etc. +It allows for grouping of tests / suites using various categorization and place tests / suites in multiple buckets. Same tests can be grouped with other tests based on the functionality , frequency, type of output etc. -e.q. +e.g. ```sql --%tags(batch,daily,csv) @@ -1238,29 +1238,31 @@ e.q. or ```sql ---%tags(api,online,json) +--%tags(online,json) +--%tags(api) ``` +Tags are defined as a comma separated list within the `--%tags` annotation. +When executing a test run with tag filter applied, the framework will find all tests associated with the given tags and execute them. +The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run. -Tags are defined as a coma separated list. When executing a test run with tag filter applied, framework will find all tests associated with given tags and execute them. Framework applies `OR` logic when resolving a tags so any tests / suites that match at least one tag will be included in the test run. +When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent. Parent suite tests are not executed, but a suitepath hierarchy is kept. -When a suite gets tagged all of its children will automatically inherit a tag and get executed along the parent. Parent suit tests are not executed. but a suitepath hierarchy is kept. - -Sample tag package. +Sample test suite package with tags. ```sql -create or replace package ut_sample_test IS +create or replace package ut_sample_test is --%suite(Sample Test Suite) - --%tag(suite1) + --%tags(api) --%test(Compare Ref Cursors) - --%tag(test1,sample) + --%tags(complex,fast) procedure ut_refcursors1; --%test(Run equality test) - --%tag(test2,sample) + --%tags(simple,fast) procedure ut_test; end ut_sample_test; @@ -1287,30 +1289,47 @@ end ut_sample_test; / ``` -Execution of the test is done by using a parameter `a_tags` +Execution of the test is done by using the parameter `a_tags` ```sql -select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'suite1')); -select * from table(ut.run(a_tags => 'test1,test2')); -select * from table(ut.run(a_tags => 'sample')); +select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); +``` +The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` -begin - ut.run(a_path => 'ut_sample_test',a_tags => 'suite1'); -end; -/ +```sql +select * from table(ut.run(a_tags => 'complex')); +``` +The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` -exec ut.run('ut_sample_test', a_tags => 'sample'); +```sql +select * from table(ut.run(a_tags => 'fast')); ``` +The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast` + +#### Tag naming convention + +Tags must follow the below naming convention: + +- tag is case sensitive +- tag can contain special characters like `$#/\?-!` etc. +- tag cannot be an empty string +- tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag +- tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch` +- leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names +#### Excluding tests/suites by tags -Tags should adhere to following rules: +It is possible to exclude parts of test suites with tags. +In order to do so, prefix the tag name to exclude with a `-` (dash) sign when invoking the test run. -- tags are case sensitive -- tags cannot be an empty string -- tags cannot contain spaces e.g. to create a multi-word `tag` please use underscores,dashes, dots etc. e.g. `test_of_batch` -- tags with empty spaces will be ignored during execution -- tags can contain special characters +Examples (based on above sample test suite) + +```sql +select * from table(ut.run(a_tags => 'api,fast,-complex')); +``` +The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`. +Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 9d5fc3bdd..63dc32485 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -284,8 +284,9 @@ select * from table(ut.run('hr.test_apply_bonus', a_random_test_order_seed => 30 # Run by Tags -In addition to the path, you can filter the tests to be run by specifying tags. Tags are defined in the test with the `--%tags`-annotation ([Read more](annotations.md#tags)). -Multiple tags are separated by comma. If multiple tags are set, all tests with __any__ of them specified are run. +In addition to the path, you can filter the tests to be run by specifying tags. Tags are defined in the test / context / suite with the `--%tags`-annotation ([Read more](annotations.md#tags)). +Multiple tags are separated by comma. +The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run. ```sql begin @@ -296,6 +297,11 @@ end; select * from table(ut.run('hr.test_apply_bonus', a_tags => 'suite1')) ``` +You can also exclude specific tags by adding a `-` (dash) in front of the tag + +```sql +select * from table(ut.run('hr.test_apply_bonus', a_tags => '-suite1')) +``` # Keeping uncommitted data after test-run diff --git a/source/core/ut_suite_builder.pkb b/source/core/ut_suite_builder.pkb index 101ab2179..111b77cbd 100644 --- a/source/core/ut_suite_builder.pkb +++ b/source/core/ut_suite_builder.pkb @@ -321,7 +321,8 @@ create or replace package body ut_suite_builder is a_procedure_name t_object_name := null ) is l_annotation_pos binary_integer; - l_tag_list ut_varchar2_list := ut_varchar2_list(); + l_tags_list ut_varchar2_list := ut_varchar2_list(); + l_tag_items ut_varchar2_list; begin l_annotation_pos := a_tags_ann_text.first; while l_annotation_pos is not null loop @@ -331,14 +332,25 @@ create or replace package body ut_suite_builder is || get_object_reference( a_suite, a_procedure_name, l_annotation_pos ) ); else - l_tag_list := l_tag_list multiset union distinct ut_utils.trim_list_elements( - ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),',') - ); + l_tag_items := ut_utils.trim_list_elements(ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),',')); + if l_tag_items is not empty then + for i in 1 .. l_tag_items.count loop + if regexp_like(l_tag_items(i),'^[^-](\S)+$') then + l_tags_list.extend(); + l_tags_list(l_tags_list.last) := l_tag_items(i); + else + a_suite.put_warning( + 'Invalid value "'||l_tag_items(i)||'" for "--%tags" annotation. See documentation for details on valid tag values. Annotation value ignored.' + || get_object_reference( a_suite, a_procedure_name, l_annotation_pos ) + ); + end if; + end loop; + end if; end if; l_annotation_pos := a_tags_ann_text.next(l_annotation_pos); end loop; - --remove empty strings from table list e.g. tag1,,tag2 and conver to rows - a_list := ut_utils.convert_collection( ut_utils.filter_list(l_tag_list,ut_utils.gc_word_no_space) ); + --remove empty strings from table list e.g. tag1,,tag2 and convert to rows + a_list := ut_utils.convert_collection( ut_utils.filter_list(set(l_tags_list),ut_utils.gc_word_no_space) ); end; procedure set_seq_no( diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index ef12e1def..f52d15bf7 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -30,8 +30,8 @@ create or replace package body ut_suite_cache_manager is and ( {:path:} and {:object_name:} and {:procedure_name:} + ) ) - ) ), {:tags:} suitepaths as ( @@ -106,8 +106,8 @@ create or replace package body ut_suite_cache_manager is function get_path_sql(a_path in varchar2) return varchar2 is begin return case when a_path is not null then q'[ - :l_path||'.' like c.path || '.%' /*all children and self*/ - or ( c.path||'.' like :l_path || '.%' --all parents + :l_path||'.' like c.path || '.%' /*all parents and self*/ + or ( c.path||'.' like :l_path || '.%' /*all children and self*/ ]' else ' :l_path is null and ( :l_path is null ' end; end; @@ -129,22 +129,31 @@ create or replace package body ut_suite_cache_manager is function get_tags_sql(a_tags_count in integer) return varchar2 is begin return case when a_tags_count > 0 then - q'[filter_tags as ( + q'[included_tags as ( select c.obj.path as path from suite_items c - where c.obj.tags multiset intersect :a_tag_list is not empty - ), + where c.obj.tags multiset intersect :a_include_tag_list is not empty or :a_include_tag_list is empty + ), + excluded_tags as ( + select c.obj.path as path + from suite_items c + where c.obj.tags multiset intersect :a_exclude_tag_list is not empty + ), suite_items_tags as ( select c.* from suite_items c where exists ( - select 1 from filter_tags t - where t.path||'.' like c.obj.path || '.%' /*all children and self*/ - or c.obj.path||'.' like t.path || '.%' --all parents + select 1 from included_tags t + where t.path||'.' like c.obj.path || '.%' /*all parents and self*/ + or c.obj.path||'.' like t.path || '.%' /*all children and self*/ + ) + and not exists ( + select 1 from excluded_tags t + where c.obj.path||'.' like t.path || '.%' /*all children and self*/ ) ),]' else - q'[dummy as (select 'x' from dual where :a_tag_list is null ),]' + q'[dummy as (select 'x' from dual where :a_include_tag_list is null and :a_include_tag_list is null and :a_exclude_tag_list is null),]' end; end; @@ -187,10 +196,23 @@ create or replace package body ut_suite_cache_manager is l_sql varchar2(32767); l_suite_item_name varchar2(20); l_tags ut_varchar2_rows := coalesce(a_tags,ut_varchar2_rows()); + l_include_tags ut_varchar2_rows; + l_exclude_tags ut_varchar2_rows; l_object_owner varchar2(250) := ut_utils.qualified_sql_name(a_object_owner); l_object_name varchar2(250) := ut_utils.qualified_sql_name(a_object_name); l_procedure_name varchar2(250) := ut_utils.qualified_sql_name(a_procedure_name); begin + + select column_value + bulk collect into l_include_tags + from table(l_tags) + where column_value not like '-%'; + + select ltrim(column_value,'-') + bulk collect into l_exclude_tags + from table(l_tags) + where column_value like '-%'; + if a_path is null and a_object_name is not null then select min(c.path) into l_path @@ -216,7 +238,7 @@ create or replace package body ut_suite_cache_manager is execute immediate l_sql bulk collect into l_results - using upper(l_object_owner), l_path, l_path, upper(a_object_name), upper(a_procedure_name), l_tags, a_random_seed; + using upper(l_object_owner), l_path, l_path, upper(a_object_name), upper(a_procedure_name), l_include_tags, l_include_tags, l_exclude_tags, a_random_seed; return l_results; end; diff --git a/test/ut3_tester/core/test_suite_builder.pkb b/test/ut3_tester/core/test_suite_builder.pkb index b49ecb2da..5f8b66ba8 100644 --- a/test/ut3_tester/core/test_suite_builder.pkb +++ b/test/ut3_tester/core/test_suite_builder.pkb @@ -1431,7 +1431,67 @@ create or replace package body test_suite_builder is '%%' ); - end; + end; + + procedure test_spaces_in_tag is + l_actual clob; + l_annotations ut3.ut_annotations; + begin + --Arrange + l_annotations := ut3.ut_annotations( + ut3.ut_annotation(2, 'suite','testsuite', null), + ut3.ut_annotation(3, 'tags',' good_tag , bad tag , good-tag ', null), + ut3.ut_annotation(8, 'test','Some test', 'test_procedure'), + ut3.ut_annotation(9, 'tags',' good_tag , bad tag , good-tag ', 'test_procedure') + ); + --Act + l_actual := invoke_builder_for_annotations(l_annotations, 'SOME_PACKAGE'); + --Assert + ut.expect(l_actual).to_be_like( + '%good_taggood-tag%'|| + 'good_taggood-tag%' + ); + ut.expect(l_actual).to_be_like( + '%Invalid value "bad tag" for "--%tags" annotation.'|| + ' See documentation for details on valid tag values. Annotation value ignored. +at package "UT3_TESTER.SOME_PACKAGE", line 3%' + ); + ut.expect(l_actual).to_be_like( + '%Invalid value "bad tag" for "--%tags" annotation.'|| + ' See documentation for details on valid tag values. Annotation value ignored. +at package "UT3_TESTER.SOME_PACKAGE.TEST_PROCEDURE", line 9%' + ); + end; + + procedure test_minus_in_tag is + l_actual clob; + l_annotations ut3.ut_annotations; + begin + --Arrange + l_annotations := ut3.ut_annotations( + ut3.ut_annotation(2, 'suite','testsuite', null), + ut3.ut_annotation(3, 'tags',' good_tag , -invalid_tag , good-tag ', null), + ut3.ut_annotation(8, 'test','Some test', 'test_procedure'), + ut3.ut_annotation(9, 'tags',' good_tag , -invalid_tag , good-tag ', 'test_procedure') + ); + --Act + l_actual := invoke_builder_for_annotations(l_annotations, 'SOME_PACKAGE'); + --Assert + ut.expect(l_actual).to_be_like( + '%good_taggood-tag%'|| + 'good_taggood-tag%' + ); + ut.expect(l_actual).to_be_like( + '%Invalid value "-invalid_tag" for "--%tags" annotation.'|| + ' See documentation for details on valid tag values. Annotation value ignored. +at package "UT3_TESTER.SOME_PACKAGE", line 3%' + ); + ut.expect(l_actual).to_be_like( + '%Invalid value "-invalid_tag" for "--%tags" annotation.'|| + ' See documentation for details on valid tag values. Annotation value ignored. +at package "UT3_TESTER.SOME_PACKAGE.TEST_PROCEDURE", line 9%' + ); + end; end test_suite_builder; / diff --git a/test/ut3_tester/core/test_suite_builder.pks b/test/ut3_tester/core/test_suite_builder.pks index 721db153f..fdf10b38d 100644 --- a/test/ut3_tester/core/test_suite_builder.pks +++ b/test/ut3_tester/core/test_suite_builder.pks @@ -5,218 +5,224 @@ create or replace package test_suite_builder is --%context(suite) --%displayname(--%suite annotation) - --%test(Sets suite name from package name and leaves description empty) - procedure no_suite_description; + --%test(Sets suite name from package name and leaves description empty) + procedure no_suite_description; - --%test(Sets suite description using first --%suite annotation) - procedure suite_description_from_suite; + --%test(Sets suite description using first --%suite annotation) + procedure suite_description_from_suite; - --%test(Gives warning if more than one --%suite annotation used) - procedure suite_annot_duplicated; + --%test(Gives warning if more than one --%suite annotation used) + procedure suite_annot_duplicated; --%endcontext --%context(displayname) --%displayname(--%displayname annotation) - --%test(Overrides suite description using first --%displayname annotation) - procedure suite_descr_from_displayname; + --%test(Overrides suite description using first --%displayname annotation) + procedure suite_descr_from_displayname; - --%test(Gives warning if more than one --%displayname annotation used) - procedure displayname_annot_duplicated; + --%test(Gives warning if more than one --%displayname annotation used) + procedure displayname_annot_duplicated; - --%test(Gives warning if --%displayname annotation has no value) - procedure displayname_annot_empty; + --%test(Gives warning if --%displayname annotation has no value) + procedure displayname_annot_empty; --%endcontext --%context(test) --%displayname(--%test annotation) - --%test(Creates a test item for procedure annotated with --%test annotation) - procedure test_annotation; + --%test(Creates a test item for procedure annotated with --%test annotation) + procedure test_annotation; - --%test(Gives warning if more than one --%test annotation used) - procedure test_annot_duplicated; + --%test(Gives warning if more than one --%test annotation used) + procedure test_annot_duplicated; - --%test(Is added to suite according to annotation order in package spec) - procedure test_annotation_ordering; + --%test(Is added to suite according to annotation order in package spec) + procedure test_annotation_ordering; --%endcontext --%context(suitepath) --%displayname(--%suitepath annotation) - --%test(Sets suite path using first --%suitepath annotation) - procedure suitepath_from_non_empty_path; + --%test(Sets suite path using first --%suitepath annotation) + procedure suitepath_from_non_empty_path; - --%test(Gives warning if more than one --%suitepath annotation used) - procedure suitepath_annot_duplicated; + --%test(Gives warning if more than one --%suitepath annotation used) + procedure suitepath_annot_duplicated; - --%test(Gives warning if --%suitepath annotation has no value) - procedure suitepath_annot_empty; + --%test(Gives warning if --%suitepath annotation has no value) + procedure suitepath_annot_empty; - --%test(Gives warning if --%suitepath annotation has invalid value) - procedure suitepath_annot_invalid_path; + --%test(Gives warning if --%suitepath annotation has invalid value) + procedure suitepath_annot_invalid_path; --%endcontext --%context(rollback) --%displayname(--%rollback annotation) - --%test(Sets rollback type using first --%rollback annotation) - procedure rollback_type_valid; + --%test(Sets rollback type using first --%rollback annotation) + procedure rollback_type_valid; - --%test(Gives warning if more than one --%rollback annotation used) - procedure rollback_type_duplicated; + --%test(Gives warning if more than one --%rollback annotation used) + procedure rollback_type_duplicated; - --%test(Gives warning if --%rollback annotation has no value) - procedure rollback_type_empty; + --%test(Gives warning if --%rollback annotation has no value) + procedure rollback_type_empty; - --%test(Gives warning if --%rollback annotation has invalid value) - procedure rollback_type_invalid; + --%test(Gives warning if --%rollback annotation has invalid value) + procedure rollback_type_invalid; --%endcontext --%context(before_after_all_each) --%displayname(--%before/after all/each annotations) - --%test(Supports multiple before/after all/each procedure level definitions) - procedure multiple_before_after; + --%test(Supports multiple before/after all/each procedure level definitions) + procedure multiple_before_after; - --%test(Supports multiple before/after all/each standalone level definitions) - procedure multiple_standalone_bef_aft; + --%test(Supports multiple before/after all/each standalone level definitions) + procedure multiple_standalone_bef_aft; - --%test(Supports mixing before/after all/each annotations on single procedure) - procedure before_after_on_single_proc; + --%test(Supports mixing before/after all/each annotations on single procedure) + procedure before_after_on_single_proc; - --%test(Supports mixed before/after all/each as standalone and procedure level definitions) - procedure multiple_mixed_bef_aft; + --%test(Supports mixed before/after all/each as standalone and procedure level definitions) + procedure multiple_mixed_bef_aft; - --%test(Gives warning if more than one --%beforeall annotation used on procedure) - procedure beforeall_annot_duplicated; + --%test(Gives warning if more than one --%beforeall annotation used on procedure) + procedure beforeall_annot_duplicated; - --%test(Gives warning if more than one --%beforeeach annotation used on procedure) - procedure beforeeach_annot_duplicated; + --%test(Gives warning if more than one --%beforeeach annotation used on procedure) + procedure beforeeach_annot_duplicated; - --%test(Gives warning if more than one --%afterall annotation used on procedure) - procedure afterall_annot_duplicated; + --%test(Gives warning if more than one --%afterall annotation used on procedure) + procedure afterall_annot_duplicated; - --%test(Gives warning if more than one --%aftereach annotation used on procedure) - procedure aftereach_annot_duplicated; + --%test(Gives warning if more than one --%aftereach annotation used on procedure) + procedure aftereach_annot_duplicated; - --%test(Gives warning on before/after all/each annotations mixed with test) - procedure before_after_mixed_with_test; + --%test(Gives warning on before/after all/each annotations mixed with test) + procedure before_after_mixed_with_test; --%endcontext --%context(context) --%displayname(--%context annotation) - --%test(Creates nested suite for content between context/endcontext annotations) - procedure suite_from_context; + --%test(Creates nested suite for content between context/endcontext annotations) + procedure suite_from_context; - --%test(Associates before/after all/each to tests in context only) - procedure before_after_in_context; + --%test(Associates before/after all/each to tests in context only) + procedure before_after_in_context; - --%test(Propagates beforeeach/aftereach to context) - procedure before_after_out_of_context; + --%test(Propagates beforeeach/aftereach to context) + procedure before_after_out_of_context; - --%test(Does not create context and gives warning when endcontext is missing) - procedure context_without_endcontext; + --%test(Does not create context and gives warning when endcontext is missing) + procedure context_without_endcontext; - --%test(Gives warning if --%endcontext is missing a preceding --%context) - procedure endcontext_without_context; + --%test(Gives warning if --%endcontext is missing a preceding --%context) + procedure endcontext_without_context; - --%test(Gives warning when two contexts have the same name and ignores duplicated context) - procedure duplicate_context_name; + --%test(Gives warning when two contexts have the same name and ignores duplicated context) + procedure duplicate_context_name; --%endcontext --%context(throws) --%displayname(--%throws annotation) - --%test(Gives warning if --%throws annotation has no value) - procedure throws_value_empty; + --%test(Gives warning if --%throws annotation has no value) + procedure throws_value_empty; - --%test(Gives warning if --%throws annotation has invalid value) - procedure throws_value_invalid; + --%test(Gives warning if --%throws annotation has invalid value) + procedure throws_value_invalid; --%endcontext --%context(beforetest_aftertest) --%displayname(--%beforetest/aftertest annotation) - --%test(Supports multiple occurrences of beforetest/aftertest for a test) - procedure before_aftertest_multi; + --%test(Supports multiple occurrences of beforetest/aftertest for a test) + procedure before_aftertest_multi; - --%test(Supports same procedure defined twice) - procedure before_aftertest_twice; + --%test(Supports same procedure defined twice) + procedure before_aftertest_twice; - --%test(Supports beforetest from external package) - procedure before_aftertest_pkg_proc; + --%test(Supports beforetest from external package) + procedure before_aftertest_pkg_proc; - --%test(Supports mix of procedure and package.procedure) - procedure before_aftertest_mixed_syntax; + --%test(Supports mix of procedure and package.procedure) + procedure before_aftertest_mixed_syntax; --%endcontext --%context(unknown_annotation) --%displayname(--%bad_annotation) - --%test(Gives warning when unknown procedure level annotation passed) - procedure test_bad_procedure_annotation; + --%test(Gives warning when unknown procedure level annotation passed) + procedure test_bad_procedure_annotation; - --%test(Gives warning when unknown package level annotation passed) - procedure test_bad_package_annotation; + --%test(Gives warning when unknown package level annotation passed) + procedure test_bad_package_annotation; --%endcontext --%context(tags_annotation) --%displayname(--%tag_annotation) - --%test(Build suite test with tag) - procedure test_tag_annotation; + --%test(Build suite test with tag) + procedure test_tag_annotation; - --%test(Build suite with tag) - procedure suite_tag_annotation; + --%test(Build suite with tag) + procedure suite_tag_annotation; - --%test(Build suite test with three tags) - procedure test_tags_annotation; - - --%test(Build suite with three tags) - procedure suite_tags_annotation; + --%test(Build suite test with three tags) + procedure test_tags_annotation; - --%test(Build suite test with two line tag annotation) - procedure test_2line_tags_annotation; - - --%test(Build suite with two line tag annotation) - procedure suite_2line_tags_annotation; + --%test(Build suite with three tags) + procedure suite_tags_annotation; - --%test(Build suite test with empty line tag annotation) - procedure test_empty_tag; - - --%test(Build suite with empty line tag annotation) - procedure suite_empty_tag; - - --%test(Build suite test with duplicate tag annotation) - procedure test_duplicate_tag; - - --%test(Build suite with duplicate tag annotation) - procedure suite_duplicate_tag; + --%test(Build suite test with two line tag annotation) + procedure test_2line_tags_annotation; - --%test(Build suite test with empty between tag annotation) - procedure test_empty_tag_between; - - --%test(Build suite with empty between tag annotation) - procedure suite_empty_tag_between; + --%test(Build suite with two line tag annotation) + procedure suite_2line_tags_annotation; + + --%test(Build suite test with empty line tag annotation) + procedure test_empty_tag; + + --%test(Build suite with empty line tag annotation) + procedure suite_empty_tag; + + --%test(Build suite test with duplicate tag annotation) + procedure test_duplicate_tag; + + --%test(Build suite with duplicate tag annotation) + procedure suite_duplicate_tag; + + --%test(Build suite test with empty between tag annotation) + procedure test_empty_tag_between; + + --%test(Build suite with empty between tag annotation) + procedure suite_empty_tag_between; + + --%test(Build suite test with special char tag annotation) + procedure test_special_char_tag; + + --%test(Build suite with special char tag annotation) + procedure suite_special_char_tag; + + --%test(Raise warning and ignore tag with spaces in tag name) + procedure test_spaces_in_tag; + + --%test(Raise warning and ignore tag starting ith '-') + procedure test_minus_in_tag; - --%test(Build suite test with special char tag annotation) - procedure test_special_char_tag; - - --%test(Build suite with special char tag annotation) - procedure suite_special_char_tag; - --%endcontext end test_suite_builder; diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 329b71357..53a819299 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1021,6 +1021,48 @@ Failures:% ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); end; + procedure tag_inc_exc_run_func_path_list is + l_results ut3.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(ut3.ut_varchar2_list(':tests.test_package_1',':tests'),a_tags => 'suite1test1,suite2test1,-suite2'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); + end; + + procedure tag_exclude_run_func_path_list is + l_results ut3.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(ut3.ut_varchar2_list(':tests,:tests2'),a_tags => '-suite1test2,-suite2test1,-test1suite3'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_3%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_3.test2%executed%' ); + end; + + procedure tag_include_exclude_run_func is + l_results ut3.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(a_tags => 'suite1,-suite1test2,-suite2test1,-test1suite3'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test2%executed%' ); + end; + procedure set_application_info is begin dbms_application_info.set_module( gc_module, gc_action ); diff --git a/test/ut3_user/api/test_ut_run.pks b/test/ut3_user/api/test_ut_run.pks index 38a6eee2c..ac2e50e25 100644 --- a/test/ut3_user/api/test_ut_run.pks +++ b/test/ut3_user/api/test_ut_run.pks @@ -175,59 +175,68 @@ create or replace package test_ut_run is --%endcontext --%context(run with tags) - --%displayname(Call ut.run with #tags) - --%beforeall(create_ut3$user#_tests) - --%afterall(drop_ut3$user#_tests) - - --%test(Execute test by tag ut_run) - procedure test_run_by_one_tag; - - --%test( Execute suite by one tag) - procedure suite_run_by_one_tag; - - --%test(Execute two tests by one tag) - procedure two_test_run_by_one_tag; - - --%test(Execute all suites tests with tag) - procedure all_suites_run_by_one_tag; - - --%test(Execute tests by passing two tags) - procedure two_test_run_by_two_tags; - - --%test(Execute suite and all of its children) - procedure suite_with_children_tag; - - --%test(Execute suite and parents) - procedure suite_with_tag_parent; - - --%test(Execute test for non existing tag) - procedure test_nonexists_tag; - - --%test(Execute test for duplicate list tags) - procedure test_duplicate_tag; - - --%test(Execute suite test for duplicate list tags) - procedure suite_duplicate_tag; - --%test(Runs given package only with package name given as path and filter by tag) - procedure run_proc_pkg_name_tag; - - --%test(Runs all from given package with package name given as path and coverage file list with tag) - procedure run_pkg_name_file_list_tag; - - --%test(Runs tests from given paths with paths list and tag) - procedure run_proc_path_list_tag; - - --%test(Runs all tests in current schema with default reporter when only tag is given) - procedure tag_run_func_no_params; + --%beforeall(create_ut3$user#_tests) + --%afterall(drop_ut3$user#_tests) + + --%test(Execute test by tag ut_run) + procedure test_run_by_one_tag; + + --%test( Execute suite by one tag) + procedure suite_run_by_one_tag; + + --%test(Execute two tests by one tag) + procedure two_test_run_by_one_tag; + + --%test(Execute all suites tests with tag) + procedure all_suites_run_by_one_tag; + + --%test(Execute tests by passing two tags) + procedure two_test_run_by_two_tags; + + --%test(Execute suite and all of its children) + procedure suite_with_children_tag; + + --%test(Execute suite and parents) + procedure suite_with_tag_parent; + + --%test(Execute test for non existing tag) + procedure test_nonexists_tag; + + --%test(Execute test for duplicate list tags) + procedure test_duplicate_tag; + + --%test(Execute suite test for duplicate list tags) + procedure suite_duplicate_tag; + + --%test(Runs given package only with package name given as path and filter by tag) + procedure run_proc_pkg_name_tag; + + --%test(Runs all from given package with package name given as path and coverage file list with tag) + procedure run_pkg_name_file_list_tag; + + --%test(Runs tests from given paths with paths list and tag) + procedure run_proc_path_list_tag; + + --%test(Runs all tests in current schema with default reporter when only tag is given) + procedure tag_run_func_no_params; + + --%test(Runs given package only with package name given as path and filter by tag) + procedure tag_run_func_pkg_name; + + --%test(Runs tests from given paths with paths list and a tag) + procedure tag_run_func_path_list; + + --%test(Runs tests from given paths with paths list and include/exclude tags) + procedure tag_inc_exc_run_func_path_list; + + --%test(Runs tests from given path and excludes specific tags) + procedure tag_exclude_run_func_path_list; - --%test(Runs given package only with package name given as path and filter by tag) - procedure tag_run_func_pkg_name; + --%test(Runs tests from given tags and exclude tags) + procedure tag_include_exclude_run_func; - --%test(Runs tests from given paths with paths list and a tag) - procedure tag_run_func_path_list; - --%endcontext --%context(ut3_info context)