Skip to content

TestSkippedException is treated as failure #2306

@Vampire

Description

@Vampire

I found the org.opentest4j.TestSkippedException as neighbor of the org.opentest4j.TestAbortedException with common parent org.opentest4j.IncompleteExecutionException.

I thought in some situations in the Spock engine it might be more appropriate to throw TestSkippedException than TestAbortedException, for example when evaluating the @IgnoreIf closure which is done right before the test itself is done.
A similar use-case would be to check assumption at the start of the test before the real test, which currently typically is done using AssumptionViolationException or TestAbortedException. But also there if the assumptions are checked in the beginning, it might semantically be more appropriate to throw a TestSkippedException instead.

From its JavaDoc I would have interpreted that TestSkippedException is meant for exactly situations I depcited above.
But if I use the TestSkippedException, JUnit treats it like any other exception that is not TestAbortedException and treats it as failure instead.

I would have expected the test to be recorded as skipped or at least as aborted, but not as failed.
Maybe it should even be
TestSkippedException => skipped
IncompleteExecutionException => aborted

Context

  • Used versions: 5.7.0-M1 / 1.7.0-M1
  • Build Tool/IDE: Gradle

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions