Skip to content

Unhandled BytesWarning when test.support.strace_helper.strace_python fails #127747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
andreas-schwab opened this issue Dec 8, 2024 · 8 comments
Closed
Labels
3.14 bugs and security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@andreas-schwab
Copy link
Contributor

andreas-schwab commented Dec 8, 2024

======================================================================
FAIL: test_all (test.test___all__.AllTest.test_all)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test___all__.py", line 129, in test_all
    self.check_all(modname)
    ~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test___all__.py", line 35, in check_all
    with warnings_helper.check_warnings(
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        (f".*{modname}", DeprecationWarning),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        ("", ResourceWarning),
        ^^^^^^^^^^^^^^^^^^^^^^
        quiet=True):
        ^^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/warnings_helper.py", line 185, in _filterwarnings
    raise AssertionError("unhandled warning %s" % reraise[0])
AssertionError: unhandled warning {message : BytesWarning('str() on a bytes instance'), category : 'BytesWarning', filename : '/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py', lineno : 85, line : None}

----------------------------------------------------------------------

Linked PRs

@eendebakpt
Copy link
Contributor

@andreas-schwab Could you provide some more details about the error? For example: which platform are you using, how did you start the tests (from the message above I gather you are running the cpython unit tests), how did you install or compile cpython, is python itself running as expected?

@eendebakpt eendebakpt added the tests Tests in the Lib/test dir label Dec 8, 2024
@andreas-schwab
Copy link
Contributor Author

andreas-schwab commented Dec 8, 2024

This is the actual test failure:

test test_subprocess crashed -- Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/libregrtest/single.py", line 181, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/libregrtest/single.py", line 128, in _load_run_test
    test_mod = importlib.import_module(module_name)
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1386, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1359, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1330, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 756, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test_subprocess.py", line 1843, in <module>
    class POSIXProcessTestCase(BaseTestCase):
    ...<1628 lines>...
                    self.assertNotRegex(non_vfork_result.event_bytes, br"(?i)vfork")
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/test_subprocess.py", line 3420, in POSIXProcessTestCase
    @strace_helper.requires_strace()
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 166, in requires_strace
    return unittest.skipUnless(_can_strace(), "Requires working strace")
                               ~~~~~~~~~~~^^
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 153, in _can_strace
    res = strace_python("import sys; sys.exit(0)", [], check=False)
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 105, in strace_python
    return _make_error("Expected strace events and exit code line",
                       stripped[-50:])
  File "/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py", line 85, in _make_error
    event_bytes=f"error({reason},details={details}) = -1".encode('utf-8'),
                                         ^^^^^^^^^
BytesWarning: str() on a bytes instance

@picnixz picnixz changed the title AssertionError: unhandled warning {message : BytesWarning('str() on a bytes instance'), category : 'BytesWarning', filename : '/home/abuild/rpmbuild/BUILD/Python-3.14.0a2/Lib/test/support/strace_helper.py', lineno : 85, line : None} Uhandled warning in test_subprocess Dec 8, 2024
@picnixz picnixz changed the title Uhandled warning in test_subprocess Unhandled BytesWarning('str() on a bytes instance') in test_subprocess Dec 8, 2024
@picnixz
Copy link
Member

picnixz commented Dec 8, 2024

Please, tell us how Python was configured and installed and how the tests were executed.

@picnixz picnixz added type-bug An unexpected behavior, bug, or error pending The issue will be closed if no feedback is provided labels Dec 8, 2024
@andreas-schwab
Copy link
Contributor Author

https://build.opensuse.org/package/live_build_log/openSUSE:Factory:RISCV/python314/standard/riscv64

./configure --host=riscv64-suse-linux --build=riscv64-suse-linux --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-platlibdir=lib64 --docdir=/usr/share/doc/packages/python --enable-ipv6 --enable-shared --with-ensurepip=no --with-system-ffi --with-system-expat --with-lto --with-ssl-default-suites=openssl --enable-optimizations --with-system-libmpdec --enable-loadable-sqlite-extensions
make
make test

@picnixz
Copy link
Member

picnixz commented Dec 8, 2024

Ok, my best guess is that the strace helper is not found, so something is raised. However, when we raise that error, we use details, which is likely a bytes instance and not a string one. Why? because details is probably obtained from stderr of subprocess.run(...) used to check the presence of strace, and we likely forgot to decode the output.

I'll check this tomorrow and provide a patch if needed.

@picnixz picnixz removed the pending The issue will be closed if no feedback is provided label Dec 8, 2024
@picnixz picnixz assigned picnixz and unassigned picnixz Dec 8, 2024
@picnixz
Copy link
Member

picnixz commented Dec 9, 2024

The error seems to be in AutoFileTests.test_syscalls_read according to your logs. Note that the test_vfork_used_when_expected is also affected and it's actually the decorator that fails.

cc @cmaloney as the one who added this decorator.

@cmaloney
Copy link
Contributor

cmaloney commented Dec 9, 2024

Walking through the warning log + code looks like the strace binary exists but gave unexpected output. That leads to a call to the _make_error helper (https://github.com/python/cpython/blob/main/Lib/test/support/strace_helper.py#L103-L109) where details is a bytes rather than an OSError or str. I'd been using f-strings to format the output, and there is an implicit conversion from bytes -> str there... Not sure if there's a common / preferred way to fix, will experiment a bit.

@andreas-schwab
Copy link
Contributor Author

No, there is no strace installed.

@picnixz picnixz added the 3.14 bugs and security fixes label Dec 10, 2024
cmaloney added a commit to cmaloney/cpython that referenced this issue Dec 10, 2024
The strace_helper code has a _make_error function to simplify making
StraceResult objects in error cases. That takes a details parameter
which is either a caught OSError or `bytes`. If it's bytes, _make_error
would implicitly coerce that to a str inside of a f-string, resulting in
 a BytesWarning.

It's useful to see if it's an OSError or bytes when debugging, resolve
by changing to format with repr().

This is an error message on an internal helper.
@picnixz picnixz changed the title Unhandled BytesWarning('str() on a bytes instance') in test_subprocess Unhandled BytesWarning('str() on a bytes instance') when test.support.strace_helper.strace_python fails Dec 10, 2024
@picnixz picnixz changed the title Unhandled BytesWarning('str() on a bytes instance') when test.support.strace_helper.strace_python fails Unhandled BytesWarning when test.support.strace_helper.strace_python` fails Dec 10, 2024
@picnixz picnixz changed the title Unhandled BytesWarning when test.support.strace_helper.strace_python` fails Unhandled BytesWarning when test.support.strace_helper.strace_python fails Dec 10, 2024
cmaloney added a commit to cmaloney/cpython that referenced this issue Dec 12, 2024
The strace_helper code has a _make_error function to simplify making
StraceResult objects in error cases. That takes a details parameter
which is either a caught OSError or `bytes`. If it's bytes, _make_error
would implicitly coerce that to a str inside of a f-string, resulting in
 a BytesWarning.

It's useful to see if it's an OSError or bytes when debugging, resolve
by changing to format with repr().

This is an error message on an internal helper.
cmaloney added a commit to cmaloney/cpython that referenced this issue Dec 12, 2024
A non-zero exit code occurs if the strace binary isn't found, and no
events will be parsed in that case (there is no output). Handle that
case by checking exit code before checking for events.

Still asserting around events rather than returning false, so that
hopefully if there's some change to `strace` that breaks the parsing,
will see that as a test failure rather than silently loosing strace
tests because they are auto-disabled.
cmaloney added a commit to cmaloney/cpython that referenced this issue Dec 12, 2024
A non-zero exit code occurs if the strace binary isn't found, and no
events will be parsed in that case (there is no output). Handle that
case by checking exit code before checking for events.

Still asserting around events rather than returning false, so that
hopefully if there's some change to `strace` that breaks the parsing,
will see that as a test failure rather than silently loosing strace
tests because they are auto-disabled.
hauntsaninja pushed a commit that referenced this issue Dec 14, 2024
The strace_helper code has a _make_error function to simplify making
StraceResult objects in error cases. That takes a details parameter
which is either a caught OSError or `bytes`. If it's bytes, _make_error
would implicitly coerce that to a str inside of a f-string, resulting in
 a BytesWarning.

It's useful to see if it's an OSError or bytes when debugging, resolve
by changing to format with repr().

This is an error message on an internal helper.

A non-zero exit code occurs if the strace binary isn't found, and no
events will be parsed in that case (there is no output). Handle that
case by checking exit code before checking for events.

Still asserting around events rather than returning false, so that
hopefully if there's some change to `strace` that breaks the parsing,
will see that as a test failure rather than silently loosing strace
tests because they are auto-disabled.
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 8, 2025
…ython#127849)

The strace_helper code has a _make_error function to simplify making
StraceResult objects in error cases. That takes a details parameter
which is either a caught OSError or `bytes`. If it's bytes, _make_error
would implicitly coerce that to a str inside of a f-string, resulting in
 a BytesWarning.

It's useful to see if it's an OSError or bytes when debugging, resolve
by changing to format with repr().

This is an error message on an internal helper.

A non-zero exit code occurs if the strace binary isn't found, and no
events will be parsed in that case (there is no output). Handle that
case by checking exit code before checking for events.

Still asserting around events rather than returning false, so that
hopefully if there's some change to `strace` that breaks the parsing,
will see that as a test failure rather than silently loosing strace
tests because they are auto-disabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants