Skip to content

Add REQUIRE macro for stopping tests early #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

pjohnmeyer
Copy link
Member

This feature, primarily authored by @paxos1977 with my collaboration, adds a new feature to UnitTest++. A check or group of checks can be marked as "required" -- triggering early termination of the test. For illustration, let's say we want to assert a pointer is non-null before accessing it.

    // REQUIRE Block Example
    Widget* widgetUnderTest = WidgetFactory.MakeMeAWidget();
    REQUIRE
    {
       CHECK(widgetUnderTest);
    }
    // the next check will never occur if widgetUnderTest is NULL
    CHECK(widgetUnderTest->WidgetId() != 0); 
    // REQUIRE Single-Line Example
    Widget* widgetUnderTest = WidgetFactory.MakeMeAWidget();
    REQUIRE CHECK(widgetUnderTest);
    // the next check will never occur if widgetUnderTest is NULL
    CHECK(widgetUnderTest->WidgetId() != 0); 

The second syntax is simpler for a single check here and there; the first is easier if you have a block of code / CHECKs you wish to require.

In this iteration, this feature is only available if exceptions are enabled (#ifndef UNITTEST_NO_EXCEPTIONS).

I still have testing to do on a few compilers, but I wanted to post it for comment while I do that.

ohz10 and others added 9 commits February 6, 2016 09:28
test when an assert fails.

The following macro has been added:

	REQUIRE

An example of when these type of checks are useful:

    std::vector<int> v = foo();

    REQUIRE(CHECK_EQUAL(3, v.size())); // test stops here on a fail
                                       // so we don't segfault looking at
                                       // v[0] below.
    CHECK_EQUAL(1, v[0]);
    CHECK_EQUAL(2, v[1]);
    CHECK_EQUAL(3, v[2]);

Multiple checks are supported as follows:

    REQUIRE({
        CHECK_EQUAL(1, 2);
        CHECK(true);
    };

In the multiple check scenario, all the checks in the REQUIRE block will
be run. After which, if any failures were reported, the TEST case will
be stopped.

When UNITTEST_NO_EXCEPTIONS is defined, REQUIRE is a noop.
I changed the definition of the REQUIRE macro to use for loops and
some comma operator shenanigans to allow things like:

    REQUIRE
    {
      CHECK(...);
      CHECK_EQUAL(..., ...);
    }

or

    REQUIRE CHECK(...);

I updated the tests and they all passed on my machine. My only
concern is that some compilers might complain about the unreachable
code in the (throw UnitTest::AssertException(), true) expression.
(1) unreachable code in for loop shenanigans is eliminated.
(2) code after a failing REQUIRE check no longer executes. Used a
    decorating TestReporter to achive this.
Rather than re-using AssertException, it felt more correct to
create a new special-purpose exception.
Was able to remove ThrowingTestReporter::SetDecorated and
::GetDecorated by changing RequiredCheckTestReporter to accept
its TestResults by reference. This simple change removed
several if-checks and some functions.
@pjohnmeyer
Copy link
Member Author

Also, the ThrowingTestReporter could be used regardless of the REQUIRE feature, if the user wished to change the behavior of all tests to stop on first failed check.

@pjohnmeyer
Copy link
Member Author

This needs a few tweaks yet; I'm having some warnings on VS due to the empty calls to UT_THROW, and I'm seeing double error messages in the standard output. I think the rethrow might need to be caught another level up.

Visual Studio 2015 complained about calling UT_THROW with zero arguments.
Visual Studio 6 complained about calling UT_CATCH with an empty second
argument. For these cases, I added UT_RETHROW(ExceptionName).

I also added a catch of RequiredCheckException to ExecuteTest to avoid
two error messages on each failed REQUIRE check.
pjohnmeyer added a commit that referenced this pull request Feb 24, 2016
…acro

Add REQUIRE macro for stopping tests early
@pjohnmeyer pjohnmeyer merged commit 983a990 into unittest-cpp:master Feb 24, 2016
@pjohnmeyer pjohnmeyer added this to the 1.5.2 / 1.6.0 milestone Feb 24, 2016
@pjohnmeyer pjohnmeyer self-assigned this Feb 24, 2016
@pjohnmeyer pjohnmeyer deleted the paxos1977-pjohnmeyer-RequiredMacro branch March 12, 2016 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants