Skip to content

Conversation

kr-2003
Copy link
Contributor

@kr-2003 kr-2003 commented Jul 8, 2025

[EDITED on 9 Jul 2025]
This PR introduces:

  1. Custom lambda function in launchExecutor.
  2. Fetching PID of the launched out-of-process(OOP) JIT executor.

This PR introduces:

  1. Pipe based redirection for stdin, stdout and stderr for out-of-process(OOP) JIT execution.
  2. Fetching PID of the launched out-of-process(OOP) JIT executor.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Jul 8, 2025
@llvmbot
Copy link
Member

llvmbot commented Jul 8, 2025

@llvm/pr-subscribers-backend-webassembly
@llvm/pr-subscribers-clang-tidy
@llvm/pr-subscribers-backend-systemz
@llvm/pr-subscribers-backend-arm
@llvm/pr-subscribers-clang-static-analyzer-1
@llvm/pr-subscribers-debuginfo
@llvm/pr-subscribers-hlsl
@llvm/pr-subscribers-clang-driver

@llvm/pr-subscribers-clang

Author: Abhinav Kumar (kr-2003)

Changes

This PR introduces:

  1. Pipe based redirection for stdin, stdout and stderr for out-of-process(OOP) JIT execution.
  2. Fetching PID of the launched out-of-process(OOP) JIT executor.

Full diff: https://github.com/llvm/llvm-project/pull/147478.diff

2 Files Affected:

  • (modified) clang/include/clang/Interpreter/RemoteJITUtils.h (+12-1)
  • (modified) clang/lib/Interpreter/RemoteJITUtils.cpp (+33-1)
diff --git a/clang/include/clang/Interpreter/RemoteJITUtils.h b/clang/include/clang/Interpreter/RemoteJITUtils.h
index 8705a3b1f669d..8fc520380dbb5 100644
--- a/clang/include/clang/Interpreter/RemoteJITUtils.h
+++ b/clang/include/clang/Interpreter/RemoteJITUtils.h
@@ -23,10 +23,13 @@
 #include <cstdint>
 #include <memory>
 #include <string>
+#include <unistd.h>
 
 llvm::Expected<std::unique_ptr<llvm::orc::SimpleRemoteEPC>>
 launchExecutor(llvm::StringRef ExecutablePath, bool UseSharedMemory,
-               llvm::StringRef SlabAllocateSizeString);
+               llvm::StringRef SlabAllocateSizeString,
+               int stdin_fd = STDIN_FILENO, int stdout_fd = STDOUT_FILENO,
+               int stderr_fd = STDERR_FILENO);
 
 /// Create a JITLinkExecutor that connects to the given network address
 /// through a TCP socket. A valid NetworkAddress provides hostname and port,
@@ -35,4 +38,12 @@ llvm::Expected<std::unique_ptr<llvm::orc::SimpleRemoteEPC>>
 connectTCPSocket(llvm::StringRef NetworkAddress, bool UseSharedMemory,
                  llvm::StringRef SlabAllocateSizeString);
 
+
+/// Returns PID of last launched executor.
+pid_t getLastLaunchedExecutorPID();
+
+/// Returns PID of nth launched executor.
+/// 1-based indexing.
+pid_t getNthLaunchedExecutorPID(int n);
+
 #endif // LLVM_CLANG_INTERPRETER_REMOTEJITUTILS_H
diff --git a/clang/lib/Interpreter/RemoteJITUtils.cpp b/clang/lib/Interpreter/RemoteJITUtils.cpp
index c0e663b764785..1b414dcd5ec25 100644
--- a/clang/lib/Interpreter/RemoteJITUtils.cpp
+++ b/clang/lib/Interpreter/RemoteJITUtils.cpp
@@ -33,6 +33,8 @@
 using namespace llvm;
 using namespace llvm::orc;
 
+static std::vector<pid_t> LaunchedExecutorPID;
+
 Expected<uint64_t> getSlabAllocSize(StringRef SizeString) {
   SizeString = SizeString.trim();
 
@@ -91,7 +93,8 @@ createSharedMemoryManager(SimpleRemoteEPC &SREPC,
 
 Expected<std::unique_ptr<SimpleRemoteEPC>>
 launchExecutor(StringRef ExecutablePath, bool UseSharedMemory,
-               llvm::StringRef SlabAllocateSizeString) {
+               llvm::StringRef SlabAllocateSizeString, int stdin_fd,
+               int stdout_fd, int stderr_fd) {
 #ifndef LLVM_ON_UNIX
   // FIXME: Add support for Windows.
   return make_error<StringError>("-" + ExecutablePath +
@@ -134,6 +137,23 @@ launchExecutor(StringRef ExecutablePath, bool UseSharedMemory,
     close(ToExecutor[WriteEnd]);
     close(FromExecutor[ReadEnd]);
 
+    if (stdin_fd != STDIN_FILENO) {
+      dup2(stdin_fd, STDIN_FILENO);
+      close(stdin_fd);
+    }
+
+    if (stdout_fd != STDOUT_FILENO) {
+      dup2(stdout_fd, STDOUT_FILENO);
+      close(stdout_fd);
+      setvbuf(stdout, NULL, _IONBF, 0);
+    }
+
+    if (stderr_fd != STDERR_FILENO) {
+      dup2(stderr_fd, STDERR_FILENO);
+      close(stderr_fd);
+      setvbuf(stderr, NULL, _IONBF, 0);
+    }
+
     // Execute the child process.
     std::unique_ptr<char[]> ExecutorPath, FDSpecifier;
     {
@@ -155,6 +175,8 @@ launchExecutor(StringRef ExecutablePath, bool UseSharedMemory,
              << ExecutorPath.get() << "\"\n";
       exit(1);
     }
+  } else {
+    LaunchedExecutorPID.push_back(ChildPID);
   }
   // else we're the parent...
 
@@ -265,3 +287,13 @@ connectTCPSocket(StringRef NetworkAddress, bool UseSharedMemory,
       std::move(S), *SockFD, *SockFD);
 #endif
 }
+
+pid_t getLastLaunchedExecutorPID() { 
+  if(!LaunchedExecutorPID.size()) return -1;
+  return LaunchedExecutorPID.back(); 
+}
+
+pid_t getNthLaunchedExecutorPID(int n) { 
+  if (n - 1 < 0 || n - 1 >= static_cast<int>(LaunchedExecutorPID.size())) return -1;
+  return LaunchedExecutorPID.at(n - 1); 
+}

Copy link

github-actions bot commented Jul 8, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@kr-2003 kr-2003 changed the title [Clang-Repl] Add pipe-based redirection and fetch PID of launched executor [Clang-Repl] Add custom function as lambda in launchExecutor and fetch PID of launched executor Jul 9, 2025
@vgvassilev
Copy link
Contributor

We will probably need a test in clang/unittest/Interpreter

@kr-2003
Copy link
Contributor Author

kr-2003 commented Jul 10, 2025

We will probably need a test in clang/unittest/Interpreter

Ok. So, replicating clang-repl's way of creating interpreter when out-of-process should work, right?

@kr-2003 kr-2003 marked this pull request as draft July 29, 2025 03:33
@kr-2003 kr-2003 changed the title [Clang-Repl] Add custom function as lambda in launchExecutor and fetch PID of launched executor [wip][Clang-Repl] Add custom function as lambda in launchExecutor and fetch PID of launched executor Jul 29, 2025
Copy link
Contributor

@Vipul-Cariappa Vipul-Cariappa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Please address the comments I have left.
I guess, you can remove the [wip] and draft status.

@kr-2003 kr-2003 changed the title [wip][Clang-Repl] Add custom function as lambda in launchExecutor and fetch PID of launched executor [Clang-Repl] Add custom function as lambda in launchExecutor and fetch PID of launched executor Jul 31, 2025
@kr-2003 kr-2003 marked this pull request as ready for review July 31, 2025 05:06
Copy link
Member

@anutosh491 anutosh491 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay to me, let's see what other reviewers have to say.

@kr-2003 kr-2003 requested a review from vgvassilev August 3, 2025 07:09
Copy link
Contributor

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@anutosh491 anutosh491 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks !

@llvmbot llvmbot added backend:PowerPC backend:Sparc backend:SystemZ backend:WebAssembly backend:X86 clangd clang-tidy clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang-format clang:headers Headers provided by Clang, e.g. for intrinsics clang:modules C++20 modules and Clang Header Modules clang:codegen IR generation bugs: mangling, exceptions, etc. clang:as-a-library libclang and C++ API clang:static analyzer debuginfo BOLT backend:DirectX HLSL HLSL Language Support backend:SPIR-V coroutines C++20 coroutines github:workflow clang:analysis clang:openmp OpenMP related changes to Clang ClangIR Anything related to the ClangIR project clang:bytecode Issues for the clang bytecode constexpr interpreter labels Aug 7, 2025
@kr-2003 kr-2003 deleted the redirection branch August 7, 2025 17:27
@kr-2003 kr-2003 restored the redirection branch August 7, 2025 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:AArch64 backend:AMDGPU backend:ARM backend:DirectX backend:MIPS backend:PowerPC backend:RISC-V backend:Sparc backend:SPIR-V backend:SystemZ backend:WebAssembly backend:X86 BOLT clang:analysis clang:as-a-library libclang and C++ API clang:bytecode Issues for the clang bytecode constexpr interpreter clang:codegen IR generation bugs: mangling, exceptions, etc. clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:headers Headers provided by Clang, e.g. for intrinsics clang:modules C++20 modules and Clang Header Modules clang:openmp OpenMP related changes to Clang clang:static analyzer clang Clang issues not falling into any other category clang-format clang-tidy clang-tools-extra clangd ClangIR Anything related to the ClangIR project coroutines C++20 coroutines debuginfo github:workflow HLSL HLSL Language Support libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants