Skip to content

Add support for contex/sub-suite #244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jgebal opened this issue Mar 25, 2017 · 6 comments
Closed

Add support for contex/sub-suite #244

jgebal opened this issue Mar 25, 2017 · 6 comments
Milestone

Comments

@jgebal
Copy link
Member

jgebal commented Mar 25, 2017

As I build my tests, I'd like to be able to group all of tests for one procedure in a package.
So that all tests for one procedure are grouped in a context/sub-suite within test package.

The values of that would be:

  • all tests for single procedure can take benefit of common beforeall/afterall
  • all tests for single procedure are nicely nested and grouped together
  • all tests for single procedure are reported in a shared group
  • it's easier to identify tests for procedure
  • test procedure names don't need to reference tested procedure name, they can simply reference the tested behaviour/functionality
@Pazus
Copy link
Member

Pazus commented Mar 26, 2017

The issue is for nested suites, right?

@jgebal
Copy link
Member Author

jgebal commented Mar 27, 2017

yes

@jgebal
Copy link
Member Author

jgebal commented May 30, 2017

It seems like a valuable extension. Currently when your PROD package has 20 procedures, and your test package has 200 tests (10 per procedure) it might be hard to navigate and hard to read.
Having test package annotated per tested procedure (to create a nested suite) could be very helpful.
Example:

package ut_depatments_api is

  --%suite(Departments api)

  --%beforeall
  procedure common_setup;

 --%context(Add departments)

  --%beforeall
  procedure add_departments_setup;

  --%test(Adds a department)
  procedure successfully_add_department;

  --%test(Raises exception when department name is NULL)
  procedure add_department_null_name;

  --%test(Raises exception when department name is duplicated)
  procedure add_department_dup_name;

  --%endcontext
  
  --%context(Remove department)

  -- and so on...

  --%endcontext
end;

@Pazus
Copy link
Member

Pazus commented Jul 6, 2017

some ideas:

  1. no nested contexts
  2. if no endcontext is present a next context finishes the previous one
  3. if a procedure is annotated with several annotations and some of them are commented out, all the outhers should be treated correctly
  4. contexts might be named, otherwise we should define the name automatically by numbering

@jgebal
Copy link
Member Author

jgebal commented Aug 27, 2017

Example:

package test_common is

  --%suite(Departments api)
  --%sutepath(org.utplsql.demo)

  --%beforeall
  procedure global_setup;

  --%test
  procedure some_test_for_global_stuff;

  --%test 
  procedure other_test_for_global_stuff;
  
end;
/
package test_depatments_api is

  --%suite(Departments api)
  --%sutepath(org.utplsql.demo.test_common)

  --%beforeall
  procedure common_setup;
  
  --%context(Add departments)

    --%beforeall
    procedure add_departments_setup;

    --%test(Adds a department)
    procedure successfully_add_department;

    --%test(Raises exception when department name is NULL)
    procedure add_department_null_name;

    --%test(Raises exception when department name is duplicated)
    procedure add_department_dup_name;

  --%endcontext
  
  --%context(Remove department)

    --%beforeall
    procedure remove_departments_setup;

    --%test(Removes a department by name)
    procedure remove_department_success;

    --%test(Fails to remove department with employees)
    procedure remove_department_failure;

  --%endcontext

  --%test(get_departments retuns all departments)
  procedure get_departments;

  --%test(update department changes deparment name)
  procedure update_department;

end;

Context can define it's own before/after blocks

When I execute: ut.run(:org.utplsql.demo) on the above, the documentation report would show something like below:

org
  utplsql
    demo
      Common stuff
        some_test_for_global_stuff [.001 sec]
        other_test_for_global_stuff [.001 sec]
        Departments api
          Add departments
            Adds a department [.001 sec]
            Raises exception when department name is NULL [.001 sec]
            Raises exception when department name is duplicated [.001 sec]
          Remove department
            Removes department by name [.001 sec]
            Fails to remove department with employees [.001 sec]
          get departments retuns all departments [.001 sec]
          update department changes deparment name [.001 sec]

Finished in 1 second
10 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)
  • ordering of tests within context can be randomized
  • ordering of items in suite can be randomized

Given the above, the test results report could also be:

org
  utplsql
    demo
      Common stuff
        some_test_for_global_stuff [.001 sec]
        Departments api
          get departments retuns all departments [.001 sec]
          Remove department
            Fails to remove department with employees [.001 sec]
            Removes department by name [.001 sec]
          Add departments
            Raises exception when department name is duplicated [.001 sec]
            Adds a department [.001 sec]
            Raises exception when department name is NULL [.001 sec]
          update department changes deparment name [.001 sec]
        other_test_for_global_stuff [.001 sec]

Finished in 1 second
10 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)

Randomization of test ordering is still possible within the bounduaries of common container.
The context would become additional (nested) container for suite items (tests and before/after blocks)

@jgebal jgebal added this to the v3.1.0 milestone Aug 31, 2017
@safokkens
Copy link

Why not add the context annotation for every procedure? The kind of grouping with context and endcontext isn't very PLSQL-esque. The position of the procedure (in the package specification at least) determines the context for which it runs.

I would rather like to see an addition next to the %test% annotation, for example a %testcontext% annotation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants