Skip to content

[Suggestion][Feedback Wanted] using cmake + google test for writing tests #1464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

yuuurchyk
Copy link
Contributor

@yuuurchyk yuuurchyk commented May 24, 2025

Hey! This is a draft suggestion PR to move to more industry-oriented approach for testing.

Suggested Changes:

  1. use cmake as a meta-build system. This should make it easier to develop new tests. Benefits:
  • no manual compilation of tests required
  • supports any compiler on any OS (as opposed to test.sh)
  1. use googletest as testing library
  • provides information about failed tests/execution time out of the box (as opposed to manual reporting)
  • we can now compile all the tests into a single executable to launch all cp-algorithms tests at once.

Side note: I was suspecting that there will be ODR violation error when including the same snippet in multiple source files, but it seems that we're good, since ODR is applicable only to a single translation unit.

Current Changes

  • rework extract_snippets.py script (add logging and command line arguments support)
  • rework the file structure in the test/ folder, suggested layout:

image

  • update 2sat and aho_korasick tests to use gtest (see ./test/src folder)

Changes To Be Implemented If This Approach Is Approved

  1. add -fsanitize=undefined -fno-sanitize-recover compiler flags
  2. update all test test_*.cpp files to use google test framework
  3. update test workflow
  4. add #pragma once to all generated snippets

Question to Maintainers

What are your thoughts on this? I'm down to rework all the test files, but would like to hear your feedback/suggestions first :)

* refactor extract_snippets.py to take arguments from command line
* add logging to extract_snippets.py script
* add suggested CMakeLists.txt
* add google test library, add example refactoring for 2sat and aho korasick tests
@yuuurchyk yuuurchyk changed the title [Suggestion] using cmake + google test for writing tests [Suggestion][Feedback Wanted] using cmake + google test for writing tests May 24, 2025
@adamant-pwn
Copy link
Member

adamant-pwn commented May 24, 2025

Hi, thank you very much for the pull request!

I agree that it would be great to rework the testing system. I would say that, ideally, we would like to use competitive verifier for this somehow, so that we refer to well-prepared problems on external judges, rather than our own ad hoc test cases. We already use it for https://lib.cp-algorithms.com, but that one is a whole library, rather than self-contained snippets.

I think we might actually try to run competitive verifier on test files that refer to snippets in a similar manner to what's going on at https://lib.cp-algorithms.com, so essentially we'd change testing files naming scheme to name.test.cpp, and then add

#define PROBLEM "..."

at the top to specify online judge that can be used for testing. Do you think it would be realistic to somehow integrate your approach with competitive verifier? Then, we would also be able to use oj-verify run ... to test things locally as well.

@yuuurchyk
Copy link
Contributor Author

yuuurchyk commented May 24, 2025

@adamant-pwn thanks for the quick feedback. I will take a look at competitive-verifier, haven't heard about it.

Will take a look at it in a week at least :(

We'll be in touch then

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants