Sorry someone broke the thread mode... Ignore my previous mail then...
With UnitTest++ I build multiple programs. Each program has a thematic collection of tests, reaching from a few like 5 to many like 500. All tests are integrated into the IDE and run as post build step. Most tests are unit tests, but often I have one collection that contain acceptance tests, these run relatively long and are not part of the normal solution/project. Theses are run on demand, rather then a post build step. If you have a naming scheme for your tests you can create a predicate and use that... (see my previous mail). Or whait until I refactored Vlad's WildcardTestPredicate (aka RegexTestPredicate) and use that. Sean On Mon, Feb 1, 2010 at 9:28 PM, David Lind <davidkl...@gmail.com> wrote: > I'm going to be creating a tutorial on > http://unittestcpp.rioki.org/tutorials soon. > > Because all unit tests get executed, by default, immediately after they are > built, were I work we needed the ability to only have the most rudimentary > tests executed at build and the rest are skipped. The more advanced tests > executed later. We came up with the following criterion: > > #define TEST_LEVEL_0 0 // Cursory tests > #define TEST_LEVEL_1 1 // Cursory tests > #define TEST_LEVEL_2 2 // Cursory tests > #define TEST_LEVEL_3 3 // Cursory tests > #define TEST_LEVEL_4 4 // Deepest, slowest, longest tests > #define TEST_LEVEL_5 5 // Fragile / driver related tests > > One of the features of my patch is the addition of TEST_LEVEL(Name, > TestLevel) and TEXT_FIXTURE_LEVEL(Fixture, Name, TestLevel) macros. > The "test level" of the TEST(Name) and TEST_FIXTURE(Fixture, Name) are 0. > > The "test level" value is specified when the unit test is written. The "run > level" is specified at run time. For example, when a "run level" value of 2 > is specified, all tests that have a "test level" value of 2 and less are > executed. All other tests are skipped. The "run level" value is a parameter > to RunAllTests(). > > --==-- > DKL > > On Mon, Feb 1, 2010 at 11:27 AM, Michael Smith <smit...@ucalgary.ca> wrote: >> >> I am not sure what this run level is -- but we have certain tests where >> in the prototype embedded system the easiest path to fast development is >> via a human intervention test -- and we need the ability to declare a >> test at a HUMAN_INVENTION_CURRENTLY_REQUIRED level and not run frequently >> >> Basically we can set a global run_level and have a macro within a test >> TEST_LEVEL(this_test_run_level) >> >> which is essentially a macro of if (this_test_run_level) < global_level { >> >> message(this test deactivated" >> return: >> >> It's actually a little more complicated than that as we do some >> interaction with a GUI controlling run-levels of individual groups of >> files >> >> Not sure if that clarifies how others might want to use a run_level >> option >> >> Mike Smith >> >> Clark Gaebel wrote: >> > What's runLevel? >> > >> > On 01/02/2010 12:23 PM, David Lind wrote: >> > >> >> I have added some command parameters To RunAllTests in my patch. They >> >> are >> >> runLevel and verboseMode. I have also added three methods, >> >> RunTestSuiteTestName, RunTestSuite, and RunTestName. >> >> >> >> >From TestRunner.h >> >> int RunAllTests(unsigned int const runLevel = 0); >> >> int RunTestSuite(char const* suiteName, unsigned int const runLevel = >> >> 0); >> >> int RunTestName(char const* testName, unsigned int const runLevel = 0); >> >> int RunTestSuiteTestName(char const* suiteName, char const* testName, >> >> unsigned int const runLevel = 0); >> >> >> >> I still need to write a tutorial for my patch. >> >> >> >> --==-- >> >> DKL >> >> >> >> On Mon, Feb 1, 2010 at 7:34 AM, Mike Smith <smit...@ucalgary.ca> wrote: >> >> >> >> >> >>> If you are planning to do that -- please think ahead about other >> >>> possible run time switches as params to Run All Tests -- it would be >> >>> easier to have a structure that could be parsed with "english" >> >>> switches >> >>> rather than just parameters that have to be placed in a particular >> >>> order. >> >>> >> >>> Even if we decided not to put in all the run-time switches in the >> >>> general UnitTest+ release, having that interface in place means that >> >>> people could customize in a common way. >> >>> >> >>> As an instructor using UnitTest++ in classes, one of the most useful >> >>> (to >> >>> the students) "run-time switch" we added to RunAllTests was the >> >>> ability >> >>> to "Show Successes" -- meaning the system reported on all asserts >> >>> that >> >>> passed -- as well as those that failed. It was a very strong support >> >>> mechanism for people just starting >> >>> >> >>> Since we are also using UnitTest++ as part of a test driven design >> >>> tool, >> >>> we also added another set of macros for expected failures XF_CHECK >> >>> for >> >>> the design situation where we (the students) had put some asserts in >> >>> -- >> >>> and then decided not to complete the implementation of the feature at >> >>> this particular time. -- we use a run time switch for the reporting >> >>> the >> >>> expected failures >> >>> >> >>> Thanks >> >>> Mike Smith >> >>> >> >>> Clark Gaebel wrote: >> >>> >> >>>> UT++ doesn't even get input from the command line afaik. We'd have to >> >>>> change the params to RunAllTests. >> >>>> >> >>>> On 31/01/2010 4:57 PM, Sean Farrell wrote: >> >>>> >> >>>> >> >>>>> Hold the phone... >> >>>>> >> >>>>> look at the implementation of: TestRunner::RunTest and >> >>>>> UNITTEST_TIME_CONSTRAINT? Is there not a possible way to implement >> >>>>> it? >> >>>>> >> >>>>> For the polymorphic part... Just instantiate the class, always or >> >>>>> once >> >>>>> for the entire run, jut don't call the methods that do the actual >> >>>>> tracking. You can also keep a boolean that can be enabled or >> >>>>> disabled >> >>>>> in the class. >> >>>>> >> >>>>> Sean >> >>>>> >> >>>>> On Sun, Jan 31, 2010 at 10:32 PM, Clark Gaebel >> >>>>> <cg.wowus...@gmail.com> >> >>>>> >> >>> wrote: >> >>> >> >>>>>> Hm, I see what you mean, but the thing is... now I've gotta get >> >>>>>> data >> >>>>>> from the command line, filter it through the levels of abstraction >> >>>>>> that >> >>>>>> are in place, and then... finally... use a polymorphic leak >> >>>>>> detector to >> >>>>>> select the right one. I don't think I'm up to the challenge. You're >> >>>>>> welcome to try though! >> >>>>>> >> >>>>>> By the way, the test fixture design is causing me headaches, I have >> >>>>>> to >> >>>>>> have duplicate code in TEST_FIXTURE_EX and in TEST_EX. >> >>>>>> >> >>>>>> I think I'll add SetUp and TearDown methods in class Test, then the >> >>>>>> fixture can implement them with the construction and destruction of >> >>>>>> the >> >>>>>> fixture. The default TEST would just replace them with null >> >>>>>> functions. >> >>>>>> >> >>>>>> Is this even do-able? Is it smart? I sure hope so! >> >>>>>> >> >>>>>> On 31/01/2010 4:27 PM, Sean Farrell wrote: >> >>>>>> >> >>>>>> >> >>>>>>> Here is the problem: >> >>>>>>> >> >>>>>>> You build UT++ one for release and debug. Put it up in binary form >> >>>>>>> for >> >>>>>>> a dozen project to use. Not all projects and developers want leak >> >>>>>>> detection to run all the time. A runtime switch, one that is given >> >>>>>>> at >> >>>>>>> RunTests like the minimum test runtime, is a good option. That >> >>>>>>> option >> >>>>>>> can the be enabled or disabled with #ifndef. >> >>>>>>> >> >>>>>>> This deployment use case is a very common one. I advise against it >> >>>>>>> and >> >>>>>>> tell people to use the vsproj files (or Makefile) so that build >> >>>>>>> settings are synced. But that means that they have to keep one >> >>>>>>> source >> >>>>>>> tree of UT per project and that is "error prone" and "wasteful". >> >>>>>>> Sooo... >> >>>>>>> >> >>>>>>> Sean >> >>>>>>> >> >>>>>>> On Sun, Jan 31, 2010 at 10:19 PM, Clark Gaebel >> >>>>>>> <cg.wowus...@gmail.com> >> >>>>>>> >> >>> wrote: >> >>> >> >>>>>>>> Ehhh, runtime switch? What happened to "simple as possible"? >> >>>>>>>> >> >>>>>>>> I like keeping all minor configurations like that in one header >> >>>>>>>> file >> >>>>>>>> called, say, config.h. >> >>>>>>>> >> >>>>>>>> I'll try out polymorphism, looks promising. >> >>>>>>>> >> >>>>>>>> On 31/01/2010 4:14 PM, Sean Farrell wrote: >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> How about you just use the same solution like TimeHelpers.*. >> >>>>>>>>> Alternatively use polymorphism: a Win32LeakDetector and a >> >>>>>>>>> NullLeakDetector class. >> >>>>>>>>> >> >>>>>>>>> Nevertheless; leak detection should be a runtime switch with no >> >>>>>>>>> overhead when off. >> >>>>>>>>> >> >>>>>>>>> Sean >> >>>>>>>>> >> >>>>>>>>> On Sun, Jan 31, 2010 at 10:08 PM, Clark Gaebel <cg.wowus.cg@ >> >>>>>>>>> >> >>> gmail.com> wrote: >> >>> >> >>> >> >>> http://github.com/wowus/unittest-cpp/commit/eede1787ab28ce420ada7b74dbd969f7a59145bd >> >>> >> >>>>>>>>>> A bit easier to use solution. >> >>>>>>>>>> >> >>>>>>>>>> Although the #ifdef hell going on in LeakDetector.cpp is >> >>>>>>>>>> killing >> >>>>>>>>>> >> >>> me. If >> >>> >> >>>>>>>>>> anyone has a better/cleaner way to do that, by all means submit >> >>>>>>>>>> a >> >>>>>>>>>> >> >>> patch. >> >>> >> >>>>>>>>>> Working on a EXPECT_LEAK macro now. >> >>>>>>>>>> >> >>>>>>>>>> On 31/01/2010 4:06 PM, Sean Farrell wrote: >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>>> On Sun, Jan 31, 2010 at 8:19 PM, Patrick Johnmeyer < >> >>>>>>>>>>> >> >>> pjohnme...@gmail.com> wrote: >> >>> >> >>>>>>>>>>>> On Sun, Jan 31, 2010 at 12:21 PM, Sean Farrell < >> >>>>>>>>>>>> >> >>> sean.farr...@rioki.org> >> >>> >> >>>>>>>>>>>> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>>> I have a better problem for you. For example you use a >> >>>>>>>>>>>>> garbage >> >>>>>>>>>>>>> collector for C/C++ like this one: >> >>>>>>>>>>>>> http://www.hpl.hp.com/personal/Hans_Boehm/gc/ The checking >> >>>>>>>>>>>>> code >> >>>>>>>>>>>>> >> >>> will >> >>> >> >>>>>>>>>>>>> flag almost all tests as memory errors. Leak detection is >> >>>>>>>>>>>>> really >> >>>>>>>>>>>>> useful, but It should be an option that is turned on (or >> >>>>>>>>>>>>> off). >> >>>>>>>>>>>>> >> >>> Not >> >>> >> >>>>>>>>>>>>> hard coded into the library. For example I don't use the >> >>>>>>>>>>>>> time >> >>>>>>>>>>>>> constants at all. Since I don't care for runtime in the >> >>>>>>>>>>>>> Tests. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> Architecturally, instantiating an object that decorates a >> >>>>>>>>>>>> >> >>> TestRunner or >> >>> >> >>>>>>>>>>>> TestReporter might be a good way to explicitly enable memory >> >>>>>>>>>>>> leak >> >>>>>>>>>>>> >> >>> detection >> >>> >> >>>>>>>>>>>> logic. >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> I don't know if I can follow you. Please elaborate on your >> >>>>>>>>>>> >> >>> proposed solution... >> >>> >> >>>>>>>>>>> Sean >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> > >> > >> > ------------------------------------------------------------------------------ >> > The Planet: dedicated and managed hosting, cloud storage, colocation >> > Stay online with enterprise data centers and the best network in the >> > business >> > Choose flexible plans and management services without long-term >> > contracts >> > Personal 24x7 support from experience hosting pros just a phone call >> > away. >> > http://p.sf.net/sfu/theplanet-com >> > _______________________________________________ >> > unittest-cpp-devel mailing list >> > unittest-cpp-devel@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/unittest-cpp-devel >> > >> > >> > >> >> -- >> Michael Smith Ph. D., >> Professor, Department of Electrical and Computer Engineering. >> Adjunct Professor, Department of Radiology, University of Calgary. >> Analog Devices University Ambassador. >> >> Electrical and Computer Engineering, Voice: (+1) 403-220-6142 >> University of Calgary, Calgary, Fax: (+1) 403-282-6855 >> Alberta, Canada T2N1N4 Email: mike.sm...@ucalgary.ca >> >> Experience can't be taught, experience can only be earned. >> >> >> >> ------------------------------------------------------------------------------ >> The Planet: dedicated and managed hosting, cloud storage, colocation >> Stay online with enterprise data centers and the best network in the >> business >> Choose flexible plans and management services without long-term contracts >> Personal 24x7 support from experience hosting pros just a phone call away. >> http://p.sf.net/sfu/theplanet-com >> _______________________________________________ >> unittest-cpp-devel mailing list >> unittest-cpp-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/unittest-cpp-devel > > > ------------------------------------------------------------------------------ > The Planet: dedicated and managed hosting, cloud storage, colocation > Stay online with enterprise data centers and the best network in the > business > Choose flexible plans and management services without long-term contracts > Personal 24x7 support from experience hosting pros just a phone call away. > http://p.sf.net/sfu/theplanet-com > _______________________________________________ > unittest-cpp-devel mailing list > unittest-cpp-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/unittest-cpp-devel > > ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ unittest-cpp-devel mailing list unittest-cpp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unittest-cpp-devel
