Skip to content

Multiple (stacked) before/after annotations #335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jgebal opened this issue May 30, 2017 · 9 comments
Closed

Multiple (stacked) before/after annotations #335

jgebal opened this issue May 30, 2017 · 9 comments

Comments

@jgebal
Copy link
Member

jgebal commented May 30, 2017

I got a request to add ability to have more than one beforetest annotation.
This way you can have multiple, named, shared setup scenarios and compose your test setup by specifying multiple beforetest annotations for a test. Some dummy example below.

package ut_departments_api is

  --%suite(Departments API)

  --%beforetest(add_employee)
  --%beforetest(add_location)
  --%beforetest(add_manager)
  --%test(Adds department)
  procedure successfully_add_department;

  -- Alternative approach could be (seems cleaner)

  --%beforetest(add_employee, add_location, add_manager)
  --%test(Adds department)
  procedure successfully_add_department;

end;
@Pazus
Copy link
Member

Pazus commented May 30, 2017

I like the second one more

@lwasylow
Copy link
Member

It's good idea however there is no reason why developer cannot wrap add_employee,add_location etc. into single package add_data and call it beforetest.

But second one looks a bit cleaner and more readable as well.

@jgebal
Copy link
Member Author

jgebal commented Jun 6, 2017

We could also allow stacked form for --%beforeall and --%beforeeach
Currently we take only the last one if several are defined in single package.
I think we should either allow all of those (stacked and ordered by position in package spec) or we should print a warning that --%beforeall was ignored(overwritten) by next declaration of --%beforeall

@jgebal jgebal added this to the v3.1.0 milestone Jun 6, 2017
@lwasylow
Copy link
Member

lwasylow commented Jun 8, 2017

I think that should be a syntax error or warning at least. If before all means to be executed before all elements of the suite then having more than one creates a bit of confusion. E.g. in which order beforeall will run and if something run before it is no longer before all.
I believe that should be allowed to be defined once only to keep the meaning of that annotation.

@jgebal
Copy link
Member Author

jgebal commented Jun 18, 2017

@lwasylow RSpec allows multiple before blocks and each of them gets executed in the order thye were defined within the context.
The beforeall means before all tests (within context), not before all items.

@jgebal
Copy link
Member Author

jgebal commented Jun 19, 2017

On the other hand, as you said, one can always stack/combine multiple procedures into --%beforeall procedure.
The multiple --%beforeall seems of much lesser value than multiple/multivalue --%beforetest
The only real value of multiple --%beforeall would be for completeness and consistency as well as to have clear and easy answer to: What if I have two --%beforeall in my test package.

Currently if you have several --%beforeall the last one defined in package spec is executed. This is a bit counter-intuitive as no warning or error is displayed.
I personally prefer to allow more than introduce limitations, just because I don't see a clear use-case for multiple --%beforeall annotations.

@jgebal
Copy link
Member Author

jgebal commented Aug 27, 2017

This is moved to separate issue: #649
So the original issue (Multiple (stacked) --%beforetest) can be more contained.

After having a bit more thought on the topic I'd like to suggest the following to the beforetest, aftertest

  • add ability to reference procedures from other packages
  • add ability to call multiple procedures

Additionally, beforeall, afterall, beforeeach, aftereach, we could be suite level annotations with similar syntax as beforetest/aftertest.
Example below:

package test_departments_api is

  --%suite(Departments API)
  --%beforeall(common_data_helper.data_setup, privileges_helper.setup)
  --%afterall(privileges_helper.cleanup, common_data_helper.data_cleanup)


  procedure setup_manager;

  --%test(Adds department)
  --%beforetest(some_package.add_employee, some_package.add_location, setup_manager)
  procedure successfully_add_department;

  --%test(Fails to add depatrment when no manager found)
  --%beforetest(common_employee_helper.setup_employee, common_location_helper.setup_location)
  procedure fail_to_add_department;

  --%test(Returns department by name)
  --%beforetest(common_department_helper.setup_depatment)
  procedure get_department_by_name;

end;

This way, we can allow for even better separation of common things.
It is still possible to achieve the same without before/after annotations referencing procedures outside of package, but it requires a lot of bolierplate code as below.

package test_departments_api is

  --%suite(Departments API)

  --%beforeall
  procedure common_setup;
  
  --%afterall
  procedure common_cleanup;

  procedure setup_manager;

  --%test(Adds department)
  --%beforetest(some_package.add_employee, some_package.add_location, setup_manager)
  procedure successfully_add_department;

  --%test(Fails to add depatrment when no manager found)
  --%beforetest(common_employee_helper.setup_employee, common_location_helper.setup_location)
  procedure fail_to_add_department;

  --%test(Returns department by name)
  --%beforetest(common_department_helper.setup_depatment)
  procedure get_department_by_name;

end;
/
...
package body test_departments_api is

  procedure common_setup is
  begin
    common_data_helper.data_setup;
    privileges_helper.setup;
  end;

  procedure common_cleanup is
  begin
    privileges_helper.cleanup;
    common_data_helper.data_cleanup;
  end;

...

Would it make sence and add value to have this kind of flexibility in annotations?

The down side I see is that we would only know at runtime if the dependencies to other packages defined by before/after are valid.

@Shoelace
Copy link
Member

i like this.. mainly because i just tried to do it and didnt understand why it didnt work

i would suggest to add it to --%beforeall also for consistency

I also like the idea of being able to refer to other packages.

@jgebal
Copy link
Member Author

jgebal commented Dec 13, 2017

I was thinking of the same

@jgebal jgebal changed the title Multiple (stacked) --%beforetest Multiple (stacked) before/after annotations Apr 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants