Skip to content

Communicate randomization seed to the compiler (Build Artifact) #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
connorkuehl opened this issue Feb 8, 2019 · 10 comments
Closed

Communicate randomization seed to the compiler (Build Artifact) #9

connorkuehl opened this issue Feb 8, 2019 · 10 comments
Assignees
@jcantrell
Labels
MVP
Milestone
Minimum Viable Pr...

Comments

@connorkuehl
Copy link

We need a way to tell Clang what our randomization seed is.
@connorkuehl connorkuehl added this to the Minimum Viable Product milestone Feb 8, 2019
@tim-pugh
Copy link
Member

#11 depends on this.

@tim-pugh tim-pugh added the MVP label Feb 12, 2019
@tim-pugh tim-pugh changed the title Communicate randomization seed to the compiler Communicate randomization seed to the compiler (Build Artifact) Feb 12, 2019
@tim-pugh tim-pugh assigned tim-pugh and jcantrell and unassigned tim-pugh Feb 12, 2019
@connorkuehl
Copy link
Author

connorkuehl commented Feb 12, 2019 •
edited by jcantrell
Loading

Acceptance criteria:

  • Generate a build artifact containing the seed so that we can reproduce builds

  • Find a way to communicate this seed to the Clang compiler

  • Incorporate the seed with randstruct. Go to each instance where we shuffle our data structures and make sure it uses a random number generator furnished with our seed.

@jcantrell
Copy link

jcantrell commented Feb 19, 2019 •
edited
Loading

Interesting methods found so far:
ParseCodeGenArgs
CreateFromArgs
ParseArgs
Clang.ConstructJobs -- handles command line arguments?

@connorkuehl
Copy link
Author

I didn't catch what file we were looking at when you shared your screen yesterday 😅

Were you looking at clang/include/clang/Driver/Options.td?

@jcantrell
Copy link

jcantrell commented Feb 19, 2019 •
edited
Loading

Well I feel sheepish, I hadn't noticed that file since I was just poking around in the debugger. Do you have any insight on how these .td definitions are used in the actual code? I'm kind of out of my depth here.
IE, how did you go about finding SemaDeclAttr.cpp and figuring out that that's where the attributes defined in the .td file are handled?

@connorkuehl
Copy link
Author

Do you have any insight on how these .td definitions are used in the actual code?

Not concretely, I think that clang/lib/Basic/Warnings.cpp is an example of usage, but this is just from a quick glance after grepping.

how did you go about finding SemaDeclAttr.cpp and figuring out that that's where the attributes defined in the .td file are handled

This was a lucky case. It was documented in the Clang CFE Internals Manual.

@jcantrell
Copy link

So far I've been looking at how "-fmessage-length" is handled, because it takes its own argument, much like our "-randsturct-seed" will. Still figuring out how the numerical argument is saved. Relevant ag:
$ ag fmessage_length
clang/include/clang/Driver/CC1Options.td
423:def fmessage_length : Separate<["-"], "fmessage-length">, MetaVarName<"">,

clang/include/clang/Driver/Options.td
1260:def fmessage_length_EQ : Joined<["-"], "fmessage-length=">, Group<f_Group>;

clang/lib/Frontend/CompilerInvocation.cpp
1550: Opts.MessageLength = getLastArgIntValue(Args, OPT_fmessage_length, 0, Diags);

clang/lib/Driver/ToolChains/Clang.cpp
4391: if (Arg *A = Args.getLastArg(options::OPT_fmessage_length_EQ)) {

@jcantrell
Copy link

Current status: I have figure out how to add a switch, just not an argument to a switch

@connorkuehl
Copy link
Author

Would it be feasible to add your switch into clang/include/clang/Driver/Options.td and then handle it in CompilerInvocation::ParseFrontendArgs()?

In that method, would you be able to retrieve the argument like they do here?

After getting the value of the argument, we'd need some way to store it in our Randstruct class. 🤔 We could probably employ the use of a global, lazy-loaded singleton for our Randstruct class in this case. So once we have the argument, we summon the singleton, give it the seed, and then when Randstruct fires up, Randstruct can ask the Singleton what the seed is. Or some creative re-imagining of this.

Or, better yet, if C++ has some way of having a const that behaves like Java's final to where it can be uninitialized at first but once it is initialized it is initialized once and only once?

Well that turned in to a rambling stream of consciousness. I hope it's at least somewhat helpful.

@jcantrell
Copy link

Status up date: i think i know what to do after it's handed off to the child process, my hangup is doing that handoff. In short, finding how/ where do i add it to Command.Arguments?

@connorkuehl connorkuehl mentioned this issue Feb 25, 2019
Open
@jcantrell jcantrell mentioned this issue Feb 26, 2019
Merged
connorkuehl pushed a commit that referenced this issue Mar 8, 2019
@hctim
Rollback of rL355585.
92dd321 
Introduces memory leak in FunctionTest.GetPointerAlignment that breaks sanitizer buildbots:

```
=================================================================
==2453==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x610428 in operator new(unsigned long) /b/sanitizer-x86_64-linux-bootstrap/build/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:105
    llvm#1 0x16936bc in llvm::User::operator new(unsigned long) /b/sanitizer-x86_64-linux-bootstrap/build/llvm/lib/IR/User.cpp:151:19
    #2 0x7c3fe9 in Create /b/sanitizer-x86_64-linux-bootstrap/build/llvm/include/llvm/IR/Function.h:144:12
    #3 0x7c3fe9 in (anonymous namespace)::FunctionTest_GetPointerAlignment_Test::TestBody() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/unittests/IR/FunctionTest.cpp:136
    #4 0x1a836a0 in HandleExceptionsInMethodIfSupported<testing::Test, void> /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc
    #5 0x1a836a0 in testing::Test::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2474
    #6 0x1a85c55 in testing::TestInfo::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2656:11
    #7 0x1a870d0 in testing::TestCase::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2774:28
    #8 0x1aa5b84 in testing::internal::UnitTestImpl::RunAllTests() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:4649:43
    #9 0x1aa4d30 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc
    #10 0x1aa4d30 in testing::UnitTest::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:4257
    #11 0x1a6b656 in RUN_ALL_TESTS /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/include/gtest/gtest.h:2233:46
    #12 0x1a6b656 in main /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/UnitTestMain/TestMain.cpp:50
    #13 0x7f5af37a22e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

Indirect leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x610428 in operator new(unsigned long) /b/sanitizer-x86_64-linux-bootstrap/build/llvm/projects/compiler-rt/lib/asan/asan_new_delete.cc:105
    llvm#1 0x151be6b in make_unique<llvm::ValueSymbolTable> /b/sanitizer-x86_64-linux-bootstrap/build/llvm/include/llvm/ADT/STLExtras.h:1349:29
    #2 0x151be6b in llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, unsigned int, llvm::Twine const&, llvm::Module*) /b/sanitizer-x86_64-linux-bootstrap/build/llvm/lib/IR/Function.cpp:241
    #3 0x7c4006 in Create /b/sanitizer-x86_64-linux-bootstrap/build/llvm/include/llvm/IR/Function.h:144:16
    #4 0x7c4006 in (anonymous namespace)::FunctionTest_GetPointerAlignment_Test::TestBody() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/unittests/IR/FunctionTest.cpp:136
    #5 0x1a836a0 in HandleExceptionsInMethodIfSupported<testing::Test, void> /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc
    #6 0x1a836a0 in testing::Test::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2474
    #7 0x1a85c55 in testing::TestInfo::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2656:11
    #8 0x1a870d0 in testing::TestCase::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:2774:28
    #9 0x1aa5b84 in testing::internal::UnitTestImpl::RunAllTests() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:4649:43
    #10 0x1aa4d30 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc
    #11 0x1aa4d30 in testing::UnitTest::Run() /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/src/gtest.cc:4257
    #12 0x1a6b656 in RUN_ALL_TESTS /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/googletest/include/gtest/gtest.h:2233:46
    #13 0x1a6b656 in main /b/sanitizer-x86_64-linux-bootstrap/build/llvm/utils/unittest/UnitTestMain/TestMain.cpp:50
    #14 0x7f5af37a22e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

SUMMARY: AddressSanitizer: 168 byte(s) leaked in 2 allocation(s).
```

See http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/11358/steps/check-llvm%20asan/logs/stdio for more information.

Also introduces use-of-uninitialized-value in ConstantsTest.FoldGlobalVariablePtr:
```
==7070==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x14e703c in User /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/User.h:79:5
    llvm#1 0x14e703c in Constant /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/Constant.h:44
    #2 0x14e703c in llvm::GlobalValue::GlobalValue(llvm::Type*, llvm::Value::ValueTy, llvm::Use*, unsigned int, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, unsigned int) /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/GlobalValue.h:78
    #3 0x14e5467 in GlobalObject /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/GlobalObject.h:34:9
    #4 0x14e5467 in llvm::GlobalVariable::GlobalVariable(llvm::Type*, bool, llvm::GlobalValue::LinkageTypes, llvm::Constant*, llvm::Twine const&, llvm::GlobalValue::ThreadLocalMode, unsigned int, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/Globals.cpp:314
    #5 0x6938f1 in llvm::(anonymous namespace)::ConstantsTest_FoldGlobalVariablePtr_Test::TestBody() /b/sanitizer-x86_64-linux-fast/build/llvm/unittests/IR/ConstantsTest.cpp:565:18
    #6 0x1a240a1 in HandleExceptionsInMethodIfSupported<testing::Test, void> /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc
    #7 0x1a240a1 in testing::Test::Run() /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc:2474
    #8 0x1a26d26 in testing::TestInfo::Run() /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc:2656:11
    #9 0x1a2815f in testing::TestCase::Run() /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc:2774:28
    #10 0x1a43de8 in testing::internal::UnitTestImpl::RunAllTests() /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc:4649:43
    #11 0x1a42c47 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc
    #12 0x1a42c47 in testing::UnitTest::Run() /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/src/gtest.cc:4257
    #13 0x1a0dfba in RUN_ALL_TESTS /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/googletest/include/gtest/gtest.h:2233:46
    #14 0x1a0dfba in main /b/sanitizer-x86_64-linux-fast/build/llvm/utils/unittest/UnitTestMain/TestMain.cpp:50
    #15 0x7f2081c412e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
    #16 0x4dff49 in _start (/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/unittests/IR/IRTests+0x4dff49)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/IR/User.h:79:5 in User
```

See http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/30222/steps/check-llvm%20msan/logs/stdio for more information.

llvm-svn: 355616
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jcantrell jcantrell

Labels
MVP
Projects
None yet
Milestone
Minimum Viable Product
Development

No branches or pull requests
3 participants
@connorkuehl @jcantrell @tim-pugh