[Clang-Repl] Add Lambda Support, PID Retrieval, and Dynamic liborc_rt Path in Clang-Repl #155140

kr-2003 · 2025-08-24T05:03:06Z

This PR builds upon the changes introduced in #152562.

Introduces:

Custom lambda function in launchExecutor.
Fetching PID of the launched out-of-process(OOP) JIT executor.
Dynamic fetching of liborc_rt path.

…into redirection

…rection

vgvassilev · 2025-08-25T09:35:35Z

clang/tools/clang-repl/ClangRepl.cpp

+    if (::OffloadArch.empty()) {
+      ::OffloadArch = "sm_35";
    }
-    CB.SetOffloadArch(OffloadArch);
+    CB.SetOffloadArch(::OffloadArch);


Are these changes relevant to the current PR?

yes. if not done, it gives following error:

error: reference to 'OffloadArch' is ambiguous

Which compiler is this? Why it works upstream?

clang/tools/clang-repl/ClangRepl.cpp

github-actions · 2025-08-25T14:47:58Z

✅ With the latest revision this PR passed the C/C++ code formatter.

clang/tools/clang-repl/ClangRepl.cpp

vgvassilev · 2025-08-25T17:05:15Z

clang/lib/Interpreter/RemoteJITUtils.cpp

@@ -33,6 +33,10 @@
 using namespace llvm;
 using namespace llvm::orc;

+#if LLVM_ON_UNIX
+static std::vector<pid_t> LaunchedExecutorPID;


That should be a member of the Interpreter class I think.

This vector is updated in launchExecutor. And launchExecutor is called before creation of interpreter. I can define it as static but I don't think it is related enough to be in interpreter class.

We need a home for this. The process is not the right home. Either the Interpreter class or the IncrementalExecutor should be good options.

vgvassilev · 2025-08-26T17:35:06Z

clang/unittests/Interpreter/OutOfProcessInterpreterTests.cpp

+  if ((SystemTriple.isOSBinFormatELF() || SystemTriple.isOSBinFormatMachO())) {
+    std::string OOPExecutor = getExecutorPath();
+    std::string OrcRuntimePath = getOrcRuntimePath(CompilerRTPath);
+    bool UseSharedMemory = false;
+    std::string SlabAllocateSizeString = "";
+    std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC;
+    EPC = ExitOnError(launchExecutor(OOPExecutor, UseSharedMemory,
+                                     SlabAllocateSizeString,
+                                     [=] { // Lambda defined inline
+                                       auto redirect = [](int from, int to) {
+                                         if (from != to) {
+                                           dup2(from, to);
+                                           close(from);
+                                         }
+                                       };
+
+                                       redirect(0, STDIN_FILENO);
+                                       redirect(1, STDOUT_FILENO);
+                                       redirect(2, STDERR_FILENO);
+
+                                       setvbuf(stdout, nullptr, _IONBF, 0);
+                                       setvbuf(stderr, nullptr, _IONBF, 0);
+                                     }));
+    if (EPC) {
+      CB.SetTargetTriple(EPC->getTargetTriple().getTriple());
+      JB = ExitOnError(clang::Interpreter::createLLJITBuilder(std::move(EPC),
+                                                              OrcRuntimePath));
+    }
+  }
+
+  return cantFail(clang::Interpreter::create(std::move(CI), std::move(JB)));


All of that should go in Interpreter.cpp logic when IsOutOfProcess is set. This way we won't have to drag OrcRuntimePath around.

vgvassilev · 2025-08-28T07:49:43Z

clang/include/clang/Interpreter/OutOfProcessJITConfig.h

+namespace clang {
+
+/// \brief Configuration options for out-of-process JIT execution.
+struct OutOfProcessJITConfig {


That struct can go as a nested Interpreter class.

vgvassilev · 2025-08-28T07:50:36Z

clang/include/clang/Interpreter/OutOfProcessJITConfig.h

+  bool UseSharedMemory;
+
+  /// String representing the slab allocation size for memory management.
+  std::string SlabAllocateSizeString;


This should be an unsigned int I think. All of these should have good default values.

vgvassilev · 2025-08-28T07:51:24Z

clang/lib/Interpreter/Interpreter.cpp

@@ -55,6 +57,8 @@
 #include "llvm/TargetParser/Host.h"
 #include "llvm/Transforms/Utils/Cloning.h" // for CloneModule

+#include <iostream>


We probably do not need this include.

vgvassilev · 2025-08-28T07:52:21Z

clang/lib/Interpreter/Interpreter.cpp

+  std::unique_ptr<llvm::orc::ExecutorProcessControl> EPC;
+  if (OutOfProcessConfig.OOPExecutor != "") {
+    // Launch an out-of-process executor locally in a child process.
+    EPC = ExitOnErr(launchExecutor(OutOfProcessConfig.OOPExecutor,


Remember this is the library, it must not abort the process. Instead the interface should return llvm::Expected<std::unique_ptr<llvm::orc::LLJITBuilder>> and propagate the error up.

vgvassilev · 2025-08-28T07:54:05Z

clang/lib/Interpreter/Interpreter.cpp

+  llvm::SmallString<256> BasePath(llvm::sys::fs::getMainExecutable(
+      "clang-repl", reinterpret_cast<void *>(&getOrcRuntimePath)));


That does not work when we are in a library. Why do we need the BasePath at all? I'd think we can ask the TC and that is mostly it. Do you try to get somehow to the source directory?

vgvassilev · 2025-08-28T07:55:02Z

clang/lib/Interpreter/RemoteJITUtils.cpp

@@ -33,6 +33,10 @@
 using namespace llvm;
 using namespace llvm::orc;

+#if LLVM_ON_UNIX
+static std::vector<pid_t> LaunchedExecutorPID;


We need a home for this. The process is not the right home. Either the Interpreter class or the IncrementalExecutor should be good options.

vgvassilev · 2025-08-28T07:55:55Z

clang/tools/clang-repl/ClangRepl.cpp

 #include <optional>

+#include <iostream>


We don't need that.

vgvassilev · 2025-08-28T07:56:18Z

clang/tools/clang-repl/ClangRepl.cpp

+    if (::OffloadArch.empty()) {
+      ::OffloadArch = "sm_35";
    }
-    CB.SetOffloadArch(OffloadArch);
+    CB.SetOffloadArch(::OffloadArch);


Which compiler is this? Why it works upstream?

vgvassilev · 2025-08-28T07:56:46Z

clang/tools/clang-repl/ClangRepl.cpp

+  } else {
+    Interp = ExitOnErr(
+        clang::Interpreter::create(std::move(CI), OutOfProcessConfig));
+  }


kr-2003 and others added 21 commits July 29, 2025 13:34

pipes for redirection in oop jit

79849c3

Merge branch 'main' into redirection

825f005

refactoring

52b0902

refactoring

3b4fcad

commenting & refactoring

e174e98

compiler-rt conditional addition

b744a9f

Merge branch 'main' into redirection

0c6c995

removed compiler-rt dep

2cf71ed

Merge branch 'redirection' of https://github.com/kr-2003/llvm-project …

7c29008

…into redirection

separate file for oop tests

41f8e54

separate file for oop tests

4f1d203

separate file for oop tests

a6a4369

test file rename

63de886

test file rename

c24e84e

test file rename

b114bb8

resolving comments & InterpreterRemoteTest

850b953

resolving comments

956f393

resolving comments

14e5afd

Custom lambda in launchExecutor and pid retrieval

65afbff

Merge branch 'main' of https://github.com/llvm/llvm-project into redi…

7dc6590

…rection

Dynamic path resolution

095a63b

vgvassilev reviewed Aug 25, 2025

View reviewed changes

vgvassilev requested a review from lhames August 25, 2025 12:07

kr-2003 added 2 commits August 25, 2025 18:52

Dynamic path resolution using IncrementalCB

2b6dc6c

Dynamic path resolution using IncrementalCB

7ad10a6

vgvassilev reviewed Aug 25, 2025

View reviewed changes

clang/tools/clang-repl/ClangRepl.cpp Outdated Show resolved Hide resolved

vgvassilev reviewed Aug 25, 2025

View reviewed changes

Addressing reviews

6a58d5f

kr-2003 requested a review from vgvassilev August 26, 2025 15:12

vgvassilev reviewed Aug 26, 2025

View reviewed changes

Sinking JitBuilder into Interpreter

f1b9135

kr-2003 requested a review from vgvassilev August 27, 2025 16:12

vgvassilev reviewed Aug 28, 2025

View reviewed changes

		llvm::SmallString<256> BasePath(llvm::sys::fs::getMainExecutable(
		"clang-repl", reinterpret_cast<void *>(&getOrcRuntimePath)));

[Clang-Repl] Add Lambda Support, PID Retrieval, and Dynamic liborc_rt Path in Clang-Repl #155140

Are you sure you want to change the base?

[Clang-Repl] Add Lambda Support, PID Retrieval, and Dynamic liborc_rt Path in Clang-Repl #155140

Conversation

kr-2003 commented Aug 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Aug 25, 2025 •

edited

Loading