[lldb][windows] use Windows APIs to print to the console #149493

charles-zablit · 2025-07-18T10:36:44Z

This patch uses the Windows APIs to print to the Windows Console, through llvm::raw_fd_ostream.

This fixes a rendering issue where the characters defined in DiagnosticsRendering.cpp ("╰" for instance) are not rendered properly on Windows out of the box, because the default codepage is not utf-8.

This solution is based on this patch downstream.

rdar://156064500

llvmbot · 2025-07-18T10:37:14Z

@llvm/pr-subscribers-llvm-support

@llvm/pr-subscribers-lldb

Author: Charles Zablit (charles-zablit)

Changes

This patch sets the codepage of the parent Windows console to utf-8 and resets it back to the original codepage once lldb exits.

This fixes a rendering issue where the characters defined in DiagnosticsRendering.cpp ("╰" for instance) are not rendered properly on Windows out of the box, because the default codepage is not utf-8.

This solution is based on this SO thread and this patch downstream.

rdar://156064500

Full diff: https://github.com/llvm/llvm-project/pull/149493.diff

2 Files Affected:

(modified) lldb/source/Plugins/Platform/Windows/PlatformWindows.cpp (+20)
(modified) lldb/source/Plugins/Platform/Windows/PlatformWindows.h (+8)

diff --git a/lldb/source/Plugins/Platform/Windows/PlatformWindows.cpp b/lldb/source/Plugins/Platform/Windows/PlatformWindows.cpp
index c0c26cc5f1954..d3e981de81313 100644
--- a/lldb/source/Plugins/Platform/Windows/PlatformWindows.cpp
+++ b/lldb/source/Plugins/Platform/Windows/PlatformWindows.cpp
@@ -41,6 +41,10 @@ LLDB_PLUGIN_DEFINE(PlatformWindows)
 
 static uint32_t g_initialize_count = 0;
 
+#if defined(_WIN32)
+std::optional<UINT> g_prev_console_cp = std::nullopt;
+#endif
+
 PlatformSP PlatformWindows::CreateInstance(bool force,
                                            const lldb_private::ArchSpec *arch) {
   // The only time we create an instance is when we are creating a remote
@@ -98,6 +102,7 @@ void PlatformWindows::Initialize() {
     default_platform_sp->SetSystemArchitecture(HostInfo::GetArchitecture());
     Platform::SetHostPlatform(default_platform_sp);
 #endif
+    SetConsoleCodePage();
     PluginManager::RegisterPlugin(
         PlatformWindows::GetPluginNameStatic(false),
         PlatformWindows::GetPluginDescriptionStatic(false),
@@ -108,6 +113,7 @@ void PlatformWindows::Initialize() {
 void PlatformWindows::Terminate() {
   if (g_initialize_count > 0) {
     if (--g_initialize_count == 0) {
+      ResetConsoleCodePage();
       PluginManager::UnregisterPlugin(PlatformWindows::CreateInstance);
     }
   }
@@ -808,3 +814,17 @@ extern "C" {
 
   return Status();
 }
+
+void PlatformWindows::SetConsoleCodePage() {
+  #if defined(_WIN32)
+    g_prev_console_cp = GetConsoleOutputCP();
+    SetConsoleOutputCP(CP_UTF8);
+  #endif
+}
+
+void PlatformWindows::ResetConsoleCodePage() {
+  #if defined(_WIN32)
+  if (g_prev_console_cp)
+    SetConsoleOutputCP(*g_prev_console_cp);
+  #endif
+}
diff --git a/lldb/source/Plugins/Platform/Windows/PlatformWindows.h b/lldb/source/Plugins/Platform/Windows/PlatformWindows.h
index 771133f341e90..d14aa52e5e1c8 100644
--- a/lldb/source/Plugins/Platform/Windows/PlatformWindows.h
+++ b/lldb/source/Plugins/Platform/Windows/PlatformWindows.h
@@ -80,6 +80,14 @@ class PlatformWindows : public RemoteAwarePlatform {
   size_t GetSoftwareBreakpointTrapOpcode(Target &target,
                                          BreakpointSite *bp_site) override;
 
+  /// Set the current console's code page to UTF-8 and store the previous
+  /// codepage in \a g_prev_console_cp.
+  static void SetConsoleCodePage();
+
+  /// Reset the current console's code page to the value stored
+  /// in \a g_prev_console_cp if any.
+  static void ResetConsoleCodePage();
+
   std::vector<ArchSpec> m_supported_architectures;
 
 private:

charles-zablit · 2025-07-18T10:37:31Z

Before

After

github-actions · 2025-07-18T10:39:11Z

✅ With the latest revision this PR passed the C/C++ code formatter.

DavidSpickett · 2025-07-18T10:50:43Z

I opened an issue for this #142568.

Where @Nerixyz mentions that SetConsoleOutputCP might have problems in cmd.exe (which probably means conhost, the original windows terminal host, as opposed to "windows terminal", the new one).

DavidSpickett · 2025-07-18T10:52:46Z

Windows Terminal is the default on Windows 10 at least. I think buildbot launches things in conhost, but if utf-8 there was a problem for tests, we would have seen it before now.

charles-zablit · 2025-07-18T11:17:42Z

I opened an issue for this #142568.

Where @Nerixyz mentions that SetConsoleOutputCP might have problems in cmd.exe (which probably means conhost, the original windows terminal host, as opposed to "windows terminal", the new one).

From my understanding, the original issue in jq is that they did not reset the codepage after the program had exited. My patch does reset it if lldb gracefully exits.

The /utf-8 approach seems promising as well but I find it suspicious that no other project uses it.

Another temporary fix for this while we figure out a long term solution could be to force the use of the ANSI characters on windows.

charles-zablit · 2025-07-18T11:18:33Z

Windows Terminal is the default on Windows 10 at least. I think buildbot launches things in conhost, but if utf-8 there was a problem for tests, we would have seen it before now.

Maybe I misunderstood your comment, but I was able to reproduce this with the latest release of lldb in the Windows 11 terminal.

DavidSpickett · 2025-07-18T11:39:09Z

Yes that makes sense, Windows Terminal doesn't default to utf-8 either. I was thinking of something else.

What if we added this new code, and it tried to set utf-8 and a test relied on that. However, this is not a problem because we already test this annotation feature via conhost and it has no problems. So even if the calls do nothing, it doesn't matter.

In other words: tests on Windows aren't scraping the output of the terminal, they'll be reading strings internally and not care about the code page.

Which is a good thing.

charles-zablit · 2025-07-18T13:12:05Z

From my understanding of the thread you linked, there are 3 ways to approach this:

Switch to ASCII characters on Windows instead of the "╰" character. This is by far the easiest way to fix this specific rendering issue, but does not address the root issue. Debugging a program with non ASCII characters will break.
Set the code page when lldb starts. Reset the codepage when it exits. This used to be a no-go because it would cause some resizing in CMD.exe, but that was over 6 years ago. Terminal is the default console in Windows 11 as of 2022.
Use /execution-charset:utf-8 as @Nerixyz suggested. I will start a build of lldb with this change. If this does not have the same problems as manually setting the code page, this sounds like the most appealing solution.
(bonus) Go the Python way and build a wrapper using WriteConsoleW, which would properly address the issue. This would however require a lot of engineering (this resolved Less than ideal handling of variable names in Cyrillic alphabet #35615).

charles-zablit · 2025-07-18T14:30:45Z

I added add_compile_options(/execution-charset:utf-8) to llvm-project\lldb\CMakeLists.txt however that did not fix the issue.

Nerixyz · 2025-07-18T14:47:39Z

I added add_compile_options(/execution-charset:utf-8) to llvm-project\lldb\CMakeLists.txt however that did not fix the issue.

Then setting the code page is probably the best idea, requiring the least amount of effort.

As far as I know, WriteConsoleW is the proper way to get Unicode on the console (this resolved #35615). However, that would require a new output, because currently, everything goes through the stdout FD and the C API.

charles-zablit · 2025-07-18T15:17:43Z

As far as I know, WriteConsoleW is the proper way to get Unicode on the console (this resolved #35615). However, that would require a new output, because currently, everything goes through the stdout FD and the C API.

Thanks for clarifying, I corrected the 4th option.

2b8c692 does not look like such big of a change, is lldb's mechanism for printing so different from clang's? I can't find where a Stream actually gets "printed" to the stdout.

Nerixyz · 2025-07-18T15:37:27Z

I can't find where a Stream actually gets "printed" to the stdout.

A NativeFile stream is used, which is created here.

2b8c692 does not look like such big of a change, is lldb's mechanism for printing so different from clang's?

I agree, that would probably be of a similar size here (i.e. check for the file handle and call the windows impl if needed).

…age" This reverts commit fcec07f.

This reverts commit 77bf2b7.

charles-zablit · 2025-07-24T12:46:06Z

I can't find where a Stream actually gets "printed" to the stdout.

A NativeFile stream is used, which is created here.

2b8c692 does not look like such big of a change, is lldb's mechanism for printing so different from clang's?

I agree, that would probably be of a similar size here (i.e. check for the file handle and call the windows impl if needed).

Thanks for the information! I was able to use the Windows API and the existing code from llvm to add proper unicode support. I think this is the better solution out of the 2 options.

compnerd · 2025-07-24T18:33:03Z

lldb/source/Host/common/File.cpp

+NativeFile::NativeFile(FILE *fh, bool transfer_ownership)
+    : m_descriptor(kInvalidDescriptor), m_own_descriptor(false), m_stream(fh),
+      m_options(), m_own_stream(transfer_ownership) {
+#ifdef _WIN32
+  int fd = _fileno(fh);
+  is_windows_console =
+      ::GetFileType((HANDLE)::_get_osfhandle(fd)) == FILE_TYPE_CHAR;
+#endif
+}
+
+NativeFile::NativeFile(int fd, OpenOptions options, bool transfer_ownership)
+    : m_descriptor(fd), m_own_descriptor(transfer_ownership),
+      m_stream(kInvalidStream), m_options(options), m_own_stream(false) {
+#ifdef _WIN32
+  is_windows_console =
+      ::GetFileType((HANDLE)::_get_osfhandle(fd)) == FILE_TYPE_CHAR;
+#endif
+}


Can you not just delegate to the fd constructor? Simply inline the _fileno call in the delegation.

That isn't quite equivalent as the implementation (for better or worse), remembers whether it was constructed with a FD or a FILE* and then uses the appropriate method to implement read/write operations.

adrian-prantl · 2025-07-28T19:28:20Z

lldb/source/Host/common/File.cpp

@@ -247,6 +248,28 @@ uint32_t File::GetPermissions(Status &error) const {
  return file_stats.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO);
 }

+NativeFile::NativeFile()
+    : m_descriptor(kInvalidDescriptor), m_stream(kInvalidStream) {}


Can you replace this with an inline initializer in the header file? Then you can just = default the constructor.

Fixed, thanks!

charles-zablit · 2025-07-30T14:19:24Z

Hi @Nerixyz, is there anything else I can add to this patch to get it merged?

Nerixyz

I can't approve anything, but it works for me on Windows 11 👍

Nerixyz · 2025-07-30T15:01:46Z

llvm/include/llvm/Support/raw_ostream.h

@@ -853,4 +853,6 @@ raw_ostream &operator<<(raw_ostream &OS, const std::optional<T> &O) {

 } // end namespace llvm

+bool write_console_impl(int FD, llvm::StringRef Data);


This should probably be in some llvm namespace - not in the global one.

Fixed by using an instance of raw_fd_ostream instead.

adrian-prantl · 2025-07-30T16:53:55Z

llvm/include/llvm/Support/raw_ostream.h

@@ -853,4 +853,6 @@ raw_ostream &operator<<(raw_ostream &OS, const std::optional<T> &O) {

 } // end namespace llvm

+bool write_console_impl(int FD, llvm::StringRef Data);


This is making an internal API that only makes sense on Windows public — would be possible to just use a raw_fd_ostream in lldb::File.cpp instead?

Using an instance of raw_fd_ostream works. Thanks!

adrian-prantl · 2025-08-04T16:03:08Z

lldb/source/Host/common/File.cpp

+
+NativeFile::NativeFile(FILE *fh, bool transfer_ownership)
+    : m_stream(fh), m_own_stream(transfer_ownership) {
+#ifdef _WIN32


Could you add a comment here explains why we do something special on Windows and (just once sentence) on why/how this works?

This patch uses the Windows APIs to print to the Windows Console, through `llvm::raw_fd_ostream`. This fixes a rendering issue where the characters defined in `DiagnosticsRendering.cpp` (`"╰"` for instance) are not rendered properly on Windows out of the box, because the default codepage is not `utf-8`. This solution is based on [this patch downstream](https://github.com/swiftlang/swift/pull/40632/files#diff-e948e4bd7a601e3ca82d596058ccb39326459a4751470eec4d393adeaf516977R37-R38). rdar://156064500

slydiman · 2025-08-19T15:07:32Z

The following buildbots are broken after this patch
lldb-x86_64-win
lldb-remote-linux-win

Please take a look and fix.

slydiman · 2025-08-19T20:10:53Z

Please fix the issue or revert ASAP to make buildbots green.

…m#149493)" This reverts commit f55dc08.

charles-zablit · 2025-08-19T21:17:22Z

Please fix the issue or revert ASAP to make buildbots green.

Could not fix it quickly enough. Opened a revert PR: #154423

…9493)" (#154423) This reverts commit f55dc08 in order to fix the issue reported [here](#149493 (comment)).

…onsole (#149493)" (#154423) This reverts commit f55dc08 in order to fix the issue reported [here](llvm/llvm-project#149493 (comment)).

[windows][lldb] force the console to use a UTF-8 codepage

77bf2b7

charles-zablit requested a review from JDevlieghere as a code owner July 18, 2025 10:36

charles-zablit self-assigned this Jul 18, 2025

charles-zablit requested a review from compnerd July 18, 2025 10:36

llvmbot added the lldb label Jul 18, 2025

charles-zablit requested a review from Michael137 July 18, 2025 10:36

fixup! [windows][lldb] force the console to use a UTF-8 codepage

fcec07f

DavidSpickett changed the title ~~[windows][lldb] force the console to use a UTF-8 codepage~~ [lldb][windows] force the console to use a UTF-8 codepage Jul 18, 2025

charles-zablit added 3 commits July 24, 2025 13:41

Revert "fixup! [windows][lldb] force the console to use a UTF-8 codep…

3e58744

…age" This reverts commit fcec07f.

Revert "[windows][lldb] force the console to use a UTF-8 codepage"

01bf62f

This reverts commit 77bf2b7.

[lldb][windows] use Windows APIs to print to the console

3291838

llvmbot added the llvm:support label Jul 24, 2025

charles-zablit changed the title ~~[lldb][windows] force the console to use a UTF-8 codepage~~ [lldb][windows] use Windows APIs to print to the console Jul 24, 2025

compnerd reviewed Jul 24, 2025

View reviewed changes

adrian-prantl reviewed Jul 28, 2025

View reviewed changes

add default values to NativeFile attributes

ae790f8

Nerixyz reviewed Jul 30, 2025

View reviewed changes

adrian-prantl reviewed Jul 30, 2025

View reviewed changes

use raw_fd_ostream

7f88778

adrian-prantl reviewed Aug 4, 2025

View reviewed changes

add comment

76a8a4f

adrian-prantl approved these changes Aug 4, 2025

View reviewed changes

charles-zablit merged commit f55dc08 into llvm:main Aug 19, 2025
9 checks passed

charles-zablit mentioned this pull request Aug 19, 2025

🍒 [lldb][windows] use Windows APIs to print to the console (#149493) swiftlang/llvm-project#11219

Open

charles-zablit added a commit to charles-zablit/llvm-project that referenced this pull request Aug 19, 2025

Revert "[lldb][windows] use Windows APIs to print to the console (llv…

133c729

…m#149493)" This reverts commit f55dc08.

charles-zablit mentioned this pull request Aug 19, 2025

Revert "[lldb][windows] use Windows APIs to print to the console (#149493)" #154423

Merged

charles-zablit added a commit that referenced this pull request Aug 20, 2025

Revert "[lldb][windows] use Windows APIs to print to the console (#14…

93189ec

…9493)" (#154423) This reverts commit f55dc08 in order to fix the issue reported [here](#149493 (comment)).

		@@ -853,4 +853,6 @@ raw_ostream &operator<<(raw_ostream &OS, const std::optional<T> &O) {

		} // end namespace llvm

		bool write_console_impl(int FD, llvm::StringRef Data);

[lldb][windows] use Windows APIs to print to the console #149493

[lldb][windows] use Windows APIs to print to the console #149493

Uh oh!

Conversation

charles-zablit commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charles-zablit commented Jul 18, 2025

Before

After

Uh oh!

github-actions bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidSpickett commented Jul 18, 2025

Uh oh!

DavidSpickett commented Jul 18, 2025

Uh oh!

charles-zablit commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charles-zablit commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidSpickett commented Jul 18, 2025

Uh oh!

charles-zablit commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charles-zablit commented Jul 18, 2025

Uh oh!

Nerixyz commented Jul 18, 2025

Uh oh!

charles-zablit commented Jul 18, 2025

Uh oh!

Nerixyz commented Jul 18, 2025

Uh oh!

charles-zablit commented Jul 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charles-zablit commented Jul 30, 2025

Uh oh!

Nerixyz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

slydiman commented Aug 19, 2025

Uh oh!

slydiman commented Aug 19, 2025

Uh oh!

charles-zablit commented Aug 19, 2025

Uh oh!

Uh oh!

charles-zablit commented Jul 18, 2025 •

edited

Loading

llvmbot commented Jul 18, 2025 •

edited

Loading

github-actions bot commented Jul 18, 2025 •

edited

Loading

charles-zablit commented Jul 18, 2025 •

edited

Loading

charles-zablit commented Jul 18, 2025 •

edited

Loading

charles-zablit commented Jul 18, 2025 •

edited

Loading