Skip to content

fix(pty): close output writer before reader on Windows to unblock close #8299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 5, 2023

Conversation

mafredri
Copy link
Member

@mafredri mafredri commented Jul 3, 2023

I haven't reproduced the issue on Windows, but I'm hoping this is a fix for the following issue (stuck on calling Close):

https://github.com/coder/coder/actions/runs/5442614267/jobs/9898091930?pr=8280

@mafredri mafredri force-pushed the mafredri/fix-pty-windows-close branch 3 times, most recently from cd01574 to 7972246 Compare July 3, 2023 11:44
@mafredri mafredri force-pushed the mafredri/fix-pty-windows-close branch from 7972246 to 5770111 Compare July 3, 2023 11:48
@mafredri mafredri force-pushed the mafredri/fix-pty-windows-close branch 2 times, most recently from 08cacc8 to 690a929 Compare July 3, 2023 12:20
@mafredri mafredri force-pushed the mafredri/fix-pty-windows-close branch from 690a929 to 2bd817d Compare July 3, 2023 12:39
ret, _, err := procClosePseudoConsole.Call(uintptr(p.console))
if ret < 0 {
return xerrors.Errorf("close pseudo console: %w", err)
if ret := windows.Handle(ret); ret != windows.S_OK && ret != windows.S_FALSE {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: I contemplated ignoring ret/err here entirely, however, in my testing it seems this function usually returns S_FALSE (1). Since I can't find documentation for this behavior, we allow S_OK too. Given positive error values, limiting this to negative doesn't make sense to me: https://learn.microsoft.com/en-us/windows/win32/seccrypto/common-hresult-values

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values like 0x80004001 are negative when interpreted as a signed 32 bit integer because they are encoded in two's compliment

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S_FALSE is a "success" code, so I think it's still fine to check ret<0 for in general.

In this specific case, Windows docs say that there is no return value, so my guess is that the Windows syscall libraries, or perhaps the Windows kernel itself is writing S_FALSE for this procedure to indicate that it's not an explicit "OK" because the call has no return value.

We could just drop the whole error handling block here and cite the docs that say this procedure doesn't return any values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values like 0x80004001 are negative when interpreted as a signed 32 bit integer because they are encoded in two's compliment

True, however the return value from Call is an uintptr and this ret wasn't being cast to 32-bit. As such the ret < 0 was nonsensical. I don't know enough about Windows syscall internals to know if we can always cast it, that would assume a syscall always returns HRESULT (maybe it does).

In this specific case, Windows docs say that there is no return value, so my guess is that the Windows syscall libraries, or perhaps the Windows kernel itself is writing S_FALSE for this procedure to indicate that it's not an explicit "OK" because the call has no return value.

Sounds plausible, this value is at least not coming from Go and Windows does define success as a positive 32-bit integer.

We could just drop the whole error handling block here and cite the docs that say this procedure doesn't return any values.

I would still like to keep it, just in case. Unless we can confirm that this syscall can't fail in some way or form.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this to int32(ret) < 0 now, to be more generic and self-documenting via a new func winerrorFailed(r1 uintptr) bool function.

@mafredri mafredri marked this pull request as ready for review July 3, 2023 12:45
@mafredri mafredri requested review from spikecurtis and mtojek July 3, 2023 12:45
Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do appreciate extra comments describing the Windows behavior, as I'm not a Windows developer 👍

I left a few comments on the wording in the comments.

ret, _, err := procClosePseudoConsole.Call(uintptr(p.console))
if ret < 0 {
return xerrors.Errorf("close pseudo console: %w", err)
if ret := windows.Handle(ret); ret != windows.S_OK && ret != windows.S_FALSE {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values like 0x80004001 are negative when interpreted as a signed 32 bit integer because they are encoded in two's compliment

ret, _, err := procClosePseudoConsole.Call(uintptr(p.console))
if ret < 0 {
return xerrors.Errorf("close pseudo console: %w", err)
if ret := windows.Handle(ret); ret != windows.S_OK && ret != windows.S_FALSE {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S_FALSE is a "success" code, so I think it's still fine to check ret<0 for in general.

In this specific case, Windows docs say that there is no return value, so my guess is that the Windows syscall libraries, or perhaps the Windows kernel itself is writing S_FALSE for this procedure to indicate that it's not an explicit "OK" because the call has no return value.

We could just drop the whole error handling block here and cite the docs that say this procedure doesn't return any values.

@mafredri mafredri force-pushed the mafredri/fix-pty-windows-close branch from 110f5e2 to 3762cb8 Compare July 5, 2023 09:47
@mafredri mafredri changed the title fix(pty): consume final console frame on Windows to unblock close fix(pty): close output writer before reader on Windows to unblock close Jul 5, 2023
@mafredri mafredri requested a review from spikecurtis July 5, 2023 10:03
Copy link
Contributor

@spikecurtis spikecurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mafredri mafredri merged commit 88c35d3 into main Jul 5, 2023
@mafredri mafredri deleted the mafredri/fix-pty-windows-close branch July 5, 2023 12:25
@github-actions github-actions bot locked and limited conversation to collaborators Jul 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants