-
Notifications
You must be signed in to change notification settings - Fork 13.4k
Update GoogleTest to v1.14.0 #65823
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update GoogleTest to v1.14.0 #65823
Conversation
A few gtests were disabled by commenting out the INSTANTIATE_TEST_SUITE_P call to the tests. This causes test failures under GoogleTest v1.14.0 as by default, every TEST_P call without a corresponding INSTANTIATE_TEST_SUITE_P call will causes a failing test from GoogleTestVerification. This patch adds GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST calls to mitigate this problem.
Patch e7bd436 introduced `ImportMatrixType` and `ImportOpenCLPipe` but didn't enable them. This cases test failures under GoogleTest v1.14.0 as uninstantiated tests needs to be explicitly allowed using GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST call. This patch enables `ImportMatrixType` test and disable `ImportOpenCLPipe` test in preparation for GoogleTest v1.14.0.
This patch bumps GoogleTest to version 1.14.0
Thank you for the change!
I think we should just use the upstream style. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with @MaskRay for the style: we should minimize the diff with upstream as much as possible!
third-party/unittest/CMakeLists.txt
Outdated
ARCHIVE DESTINATION "lib${LLVM_LIBDIR_SUFFIX}" COMPONENT llvm_gtest) | ||
install(DIRECTORY googletest/include/gtest/ DESTINATION include/llvm-gtest/gtest/ COMPONENT llvm_gtest) | ||
install(DIRECTORY googlemock/include/gmock/ DESTINATION include/llvm-gmock/gmock/ COMPONENT llvm_gtest) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is undoing a change made recently in 91b3ca3. Is that intentional or was your checkout just stale?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My checkout just stales. Thanks for pointing it out.
This directory contains Google Test 1.10.0, with all elements removed except for | ||
the actual source code, to minimize the addition to the LLVM distribution. | ||
This directory contains Google Test 1.14.0, | ||
revision `f8d7d77c06936315286eb55f8de22cd23c188571`, with all elements removed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't include the revision in the googlemock
README.LLVM
file so I'd avoid it here also for consistency (or include it in the other one as well).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parts of the googletest documentation claim to follow a "live at HEAD" philosophy, so a hash reference (here and in the other README) seems appropriate. Although using a tag, if there is one, would be preferable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
v1.14.0 is a valid tag and f8d7d77c06936315286eb55f8de22cd23c188571
is the revision that it resolves to. The content matches the 1.14.0 source tar ball. The reason that I include a hash here is because the previous roll v1.10.0 doesn't have a valid tag and I had a bit hard time to find the correct revision for doing a diff to see what LLVM have patched. Since v1.14.0 is a valid tag, I think it is appropriate to remove the hash and just leave v1.14.0 in the doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice to see the LLVM intrusion into googletest proper is very minimized!
IME, testing on Linux with both gcc and clang as build compilers is very helpful. With Windows/MSVC that covers the three main cases.
@@ -10,14 +10,10 @@ Cleaned up as follows: | |||
# Remove all the unnecessary files and directories | |||
$ rm -f CMakeLists.txt configure* Makefile* CHANGES CONTRIBUTORS README README.md .gitignore | |||
$ rm -rf build-aux make msvc scripts test docs | |||
$ rm -f `find . -name \*\.pump` | |||
$ rm -f src/gmock_main.cc | |||
|
|||
# Put the license in the consistent place for LLVM. | |||
$ mv LICENSE LICENSE.TXT | |||
|
|||
Modified as follows: | |||
* Support for std::begin/std::end in gmock-matchers.h |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this still true? If it is, it sounds like the kind of patch that ought to be made in upstream googletest. If not, please remove this line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, gmock-matchers.h still have this part patched, see https://gist.github.com/zeroomega/fb24b1d1c4252b852200e15fda29384d#file-gtest_llvm_modification-patch-L251 . When locally test it, without the patch, LLVM unit tests will have build errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That seems odd, and worth following up with upstream googletest. In the meantime, I think it would be clearer to future maintainers if we did something to clearly identify the code blocks that were changed, and why. For example,
// LLVM local change to support std::begin/std::end
followed by the new code, and then the old code commented-out. That way diffs will provide clues, and merges/commits will have conflicts that someone will need to sort out properly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I put the local changes into commented blocks in change in amend commit: 2fe51a3 in this PR, please take a look.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked at that specific commit, seems fine.
$ rm -f src/gtest_main.cc | ||
|
||
# Put the license in the consistent place for LLVM. | ||
$ mv LICENSE LICENSE.TXT | ||
|
||
Modified as follows: | ||
* Added support for NetBSD, Minix and Haiku. | ||
* Added raw_os_ostream support to include/gtest/internal/custom/gtest-printers.h. | ||
* Added StringRef support to include/gtest/internal/custom/gtest-printers.h. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also include/gtest/gtest-message.h and include/gtest/gtest-printers.h. Probably worth saying something like "(see uses of llvm_gtest)" to help the next person doing a roll.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a description for the these two headers.
This directory contains Google Test 1.10.0, with all elements removed except for | ||
the actual source code, to minimize the addition to the LLVM distribution. | ||
This directory contains Google Test 1.14.0, | ||
revision `f8d7d77c06936315286eb55f8de22cd23c188571`, with all elements removed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parts of the googletest documentation claim to follow a "live at HEAD" philosophy, so a hash reference (here and in the other README) seems appropriate. Although using a tag, if there is one, would be preferable.
680d1aa
to
ac0785c
Compare
Bump googletest to v1.14.0 This patch bumps GoogleTest to version 1.14.0
Bump googletest to v1.14.0 This patch bumps GoogleTest to version 1.14.0
I have addressed all review comments in the the latest amend commits. Please take a look. |
I have locally tested this PR using gcc 12.2.0 on Linux x64 and MSVC from VS2019, both passed without issues. @pogo59 Do you have any concerns or suggestions before we merge this? |
I'm happy with this, the bots will catch any environmental issues that you haven't found yet. LGTM. |
First bot failure: https://lab.llvm.org/buildbot/#/builders/127/builds/55068
Probably caused by missing include header. This will be addressed by 063cd55. |
I'm seeing some unused functions in files that haven't been changed in years, which I'm guessing come from this update - did the new googletest change how custom output works?
I guess we can just delete this function, but I'm not sure if it means we've degraded the error messages for these tests or anything like that. |
Yes new gtest changes how customized function work. These functions either need to be deleted or revised. I plan to revert this roll as I am seeing some linker error in bots:
It will need some time to debug. |
This reverts commit 54c1a9b. It breaks a few llvm bots.
I looked into the "unused functions" issue shown up in bots like: https://lab.llvm.org/buildbot/#/builders/57/builds/29853 . The root cause is that GoogleTest changed the way it converts an arbitrary object into string. In the past, any object that cannot be printed to the string will eventually matched to function at
OS << V.V; to print the value to the stream, if the object overrides its << operator, like
RawBytesPrinter defined at
operator<< override won't be used and a unused function warnig(error) will be thrown.
So deleting these |
I look through all the bot failures from my mail box and I think so far there are two issues:
I plan to reland the roll after it passes more local tests. For the incremental build bug, I plan to manually trigger a clean build once these runtimes bots pick up my change. Please let me know if you have questions and comments. |
This patch reland 54c1a9b, which updates GoogleTest to v1.14.0. This patch fixes bots failures caused by the early patch.
Relanded as a866ce7. I will monitor the bots and manually trigger clean build if linker error appears again. |
I see compilation problems with this patch. So the problem: On a RHEL7 server we see stuff like: If we don't build builtins (LLVM_BUILTIN_TARGETS) I don't see any problems. |
Ok, seems like the only problem is the MemProf unit test So if I manually disable that I don't see any other failures with this patch. |
I think any components that uses freshly built clang (stage1) could potentially run into this problem again, the correct way to solve it is to provide a compatible C++ Library to the freshly built clang when it built compiler-rt and related tests. For example, use -D with |
Thanks! Yes, I think we hadn't considered that some things we set up for the regular build wasn't forwarded to the compiler-rt test. |
This PR updates GoogleTest to v1.14.0 . It also updates the Clang ASTTest to fix a compatibility issue with the new GoogleTest.
This reverts commit 54c1a9b. It breaks a few llvm bots.
This patch reland 54c1a9b, which updates GoogleTest to v1.14.0. This patch fixes bots failures caused by the early patch.
We're seeing unit test crashes on Windows when building with rpmalloc (via
(Our bug: https://crbug.com/1487548) |
Just tried it, but I didn't hit any asserts. |
The new implementation was brought in with the gtest update in a866ce7, but it crashes when building with rpmalloc, see #65823 (comment) Comment out the new implementation basically gives us the code before the gtest update.
We have seen occasional segfaults when using rpmalloc in other tools such as GN and Ninja which we haven't been able to get to the bottom of so we eventually switched to another allocator. It's possible that it's the same issue here, in which case perhaps we should reconsider our support for rpmalloc. |
The new implementation was brought in with the gtest update in a866ce7, but it crashes when building with rpmalloc, see llvm#65823 (comment) Comment out the new implementation basically gives us the code before the gtest update.
@zmodem I haven’t been able to repro with the recipe posted on https://crbug.com/1487548. I am using latest rpmalloc (main branch), latest VS 2022, latest WinSDK. I am running on Win11, on a AMD Ryzen9 CPU. Are you running on WinServer2022? Are able to give more precision on the conditions for reproducing? |
I believe we're on the 10.0.19041.0 (2020-04) SDK and VS 16.6.1, running on Windows 10. The rpmalloc version is bc1923f which is probably old by now. I'll try with the latest rpmalloc version, and will also see if I can dig into the stl headers a bit, but I won't be able to get to this today. |
I managed to repro. Updating to latest rpmalloc doesn't help. It is actually this, still open, issue: microsoft/STL#1066 |
This PR updates GoogleTest to v1.14.0
The LLVM specific modification over the GoogleTest can be seen from: https://gist.github.com/zeroomega/fb24b1d1c4252b852200e15fda29384d
List of removed GoogleTest files can be found:
https://gist.github.com/zeroomega/54d1da74aa5df9a22bf60474adc53cab
2 additional commits are required for the GoogleTest roll. Due to a change in how GoogleTest treats uninstantiated tests, any test added by
TEST_P
needs an correspondingINSTANTIATE_TEST_SUITE_P
, otherwise an error will be reported. However, in LLVM, sometimesINSTANTIATE_TEST_SUITE_P
calls were commented out to temporarily disable tests. In this case in v1.14.0, aGTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST
is needed to suppress the error. This macro did not exist in v1.10.0 so the change needs to land together with v1.14.0 roll to avoid build breakages.The roll was tested under Linux x64 and Windows x64 and pass all tests under testing. It would be great if we can tests it under more build variants and hosts in LLVM bots before landing it.
I admit I cannot create a shellscript to do the auto roll. At least it was not possible without some non-trival refactoring to LLVM's patch and move these LLVM's additional functions into
gtest_port.h/cc
andgmock_port.h/cc
. A better approach would be upstream LLVM's changes to GoogleTest so we can just keep a vanilla GoogleTest in LLVM (using git submodule).The GoogleTest code also uses a different code style (Google C++ style instead of LLVM style). Shall I reformat the entire GoogleTest code base?