Skip to content

openssl unsupported hash type blake2s error after building cpython 3.11.1 with openssl #101148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
nanaceba opened this issue Jan 19, 2023 · 4 comments
Closed

openssl unsupported hash type blake2s error after building cpython 3.11.1 with openssl #101148

nanaceba opened this issue Jan 19, 2023 · 4 comments
Labels
3.11 only security fixes 3.12 only security fixes build The build process and cross-build topic-SSL type-bug An unexpected behavior, bug, or error

Comments

@nanaceba
Copy link

nanaceba commented Jan 19, 2023 •
edited
Loading

Bug report

We are building and linking cpython with OpenSSL_1_1_1l using --with-openssl. Everything was working fine from 3.8.0 to 3.10.5 python. After we build 3.11.1 with same openssl version using same build code we start seeing code for hash not found error when trying to use pip

 ~/test/bin ./python3.11 -m pip install pytest
ERROR:root:code for hash blake2b was not found.
Traceback (most recent call last):
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2b
ERROR:root:code for hash blake2s was not found.
Traceback (most recent call last):
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
Collecting pytest
  Using cached pytest-7.2.1-py3-none-any.whl (317 kB)
Collecting attrs>=19.2.0
  Using cached attrs-22.2.0-py3-none-any.whl (60 kB)
Collecting iniconfig
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging
  Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting pluggy<2.0,>=0.12
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Installing collected packages: pluggy, packaging, iniconfig, attrs, pytest
Successfully installed attrs-22.2.0 iniconfig-2.0.0 packaging-23.0 pluggy-1.0.0 pytest-7.2.1

Also blake2s is printed as available

 ~/test1/bin ./python3
Python 3.11.1 (tags/v3.11.1-dirty:a7a450f, Jan 27 2023, 20:43:46) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
ERROR:root:code for hash blake2b was not found.
Traceback (most recent call last):
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2b
ERROR:root:code for hash blake2s was not found.
Traceback (most recent call last):
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eugene.sevostianov/test1/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
>>> print(hashlib.algorithms_available)
{'md5', 'sha3_256', 'blake2s', 'sha256', 'whirlpool', 'ripemd160', 'sha512_256', 'sha3_512', 'md4', 'sha512', 'sha512_224', 'mdc2', 'sha224', 'shake_256', 'sha384', 'sha1', 'blake2b', 'sha3_384', 'shake_128', 'sm3', 'sha3_224', 'md5-sha1'}

We configure and build openssl as static library and build cpython with static lib.

            ssl_builder.run_in_build(
                ("perl", "../source/Configure", f"darwin64-{ssl_builder.flavor}-cc", "no-shared",
                 f"--prefix={ssl_builder.install_dir}",
                 f"--openssldir={ssl_builder.install_dir}",
                 ), build_env=BuildEnv(cflags=(macos_version_flag,),
                                       ldflags=(macos_version_flag,)).get_modified_with_private_deps())

we build 2 versions of open ssl lib for arm and x64 and then lipo them, and feed that fat static library to python.
python configure

    config_cmd = ("bash", "../source/configure",
                  f"--prefix={cpython_builder.install_dir}",
                  f"--with-openssl={lipo_ssl_root}")

Can't find out what I am missing.

Your environment

  • CPython versions tested on: 3.11.1
  • Operating system and architecture: 13.1 mac os, arm64 m1 mac
@nanaceba nanaceba added the type-bug An unexpected behavior, bug, or error label Jan 19, 2023
@nanaceba
Copy link
Author

Hi @tiran ! Can you please take a look and maybe you have some ideas what we could look at? Seems like you an expert with openssl-python integration.

@nikhilshinday
Copy link

If anyone else had this issue on an M1 Mac, I resolved it by following instructions here:
https://stackoverflow.com/a/72508879/2309498

@arhadthedev arhadthedev added build The build process and cross-build 3.11 only security fixes topic-SSL 3.12 only security fixes OS-mac and removed OS-mac labels Feb 28, 2023
@Yhg1s
Copy link
Member

Yhg1s commented Mar 24, 2023

We provide our own implementation of blake2s/blake2b, not using the OpenSSL versions. (The OpenSSL versions are more limited but they are available as "blake2s256" and "blake2b512" if your OpenSSL supports them.) "blake2b" and "blake2s" should always be available, which is why they appear in algorithms_available. They're implemented in the separate _blake2 extension module. Something probably caused the build of _blake2 to fail on your system. (It may be caused by #102027 which has been fixed but not released yet.)

@Yhg1s Yhg1s closed this as completed Mar 24, 2023
@nanaceba
Copy link
Author

@Yhg1s can confirm, seems fixed in 3.11.3! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.11 only security fixes 3.12 only security fixes build The build process and cross-build topic-SSL type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Yhg1s @arhadthedev @nikhilshinday @nanaceba