Support build 37 and 38 rebased #257

jcfr · 2019-08-06T22:29:52Z

@dbrnz @dand-oss This PR includes you respective set of changes 🙏 and is based on current master.

I will post here comment and questions.

Ultimately, some more commit may be squashed. Also to give credit, relevant Co-authored-by: ... git message trailer will be added.

…eleases

* Rename "Modules/random.c" to "Modules/bootstrap_hash.c" See python/cpython@6b4be19 bpo-22257: Small changes for PEP 432. (#1728) PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes. * Remove bundled copy of libffi (Closes #27979) See python/cpython@f40d4dd An installed copy of libffi is now required for building _ctypes on any platform but OSX and Windows.

… module

…ndent.

jcfr · 2019-08-06T22:30:46Z

@dbrnz when integrating the change of @dand-oss in #256, you reverted python uses external libffi now, was this intentional ?

dbrnz · 2019-08-07T00:07:21Z

@jcfr reverting libffi changes wasn't intentional, so please unrevert as applicable...

dbrnz · 2019-08-08T22:38:00Z

Hmm @jcfr , commit c59e3d1 of @dand-oss (rebased as commit 77d11f7 and which I did merge), resulted in:

# get libffi
find_path(FFI_INCLUDE_DIR ffi.h)
find_library(FFI_LIBRARY NAMES ffi libffi)
add_python_extension(_ctypes
    SOURCES ${ctypes_COMMON_SOURCES}
    INCLUDEDIRS ${FFI_INCLUDE_DIR}
    LIBRARIES ${FFI_LIBRARY}
)

when not APPLE. This code is now missing as a result of commit 3554d50. Shouldn't the above apply for Python 3.7, so we end up with:

if(PY_VERSION VERSION_LESS "3.7")
    set(_libffi_sources)
    set(_libffi_include_dirs)
      .
      .
      .
    add_python_extension(_ctypes
        SOURCES ${ctypes_COMMON_SOURCES} ${_libffi_sources}
        ${_libffi_include_dirs}
    )
else()
    find_path(FFI_INCLUDE_DIR ffi.h)
    find_library(FFI_LIBRARY NAMES ffi libffi)
    add_python_extension(_ctypes
       SOURCES ${ctypes_COMMON_SOURCES}
       INCLUDEDIRS ${FFI_INCLUDE_DIR}
       LIBRARIES ${FFI_LIBRARY}
    )
endif()

jcfr · 2019-08-09T01:19:59Z

Good catch. That would indeed make more sense.

…

On Thu, Aug 8, 2019, 6:38 PM David Brooks ***@***.***> wrote: Hmm @jcfr <https://github.com/jcfr> , commit c59e3d1 <c59e3d1> of @dand-oss <https://github.com/dand-oss> (rebased as commit 77d11f7 <77d11f7> and which I did merge), resulted in: # get libffi find_path(FFI_INCLUDE_DIR ffi.h) find_library(FFI_LIBRARY NAMES ffi libffi) add_python_extension(_ctypes SOURCES ${ctypes_COMMON_SOURCES} INCLUDEDIRS ${FFI_INCLUDE_DIR} LIBRARIES ${FFI_LIBRARY} ) when not APPLE. This code is now missing as a result of commit 3554d50 <3554d50>. Shouldn't the above apply for Python 3.7, so we end up with: if(PY_VERSION VERSION_LESS "3.7") set(_libffi_sources) set(_libffi_include_dirs) . . . add_python_extension(_ctypes SOURCES ${ctypes_COMMON_SOURCES} ${_libffi_sources} ${_libffi_include_dirs} ) else() find_path(FFI_INCLUDE_DIR ffi.h) find_library(FFI_LIBRARY NAMES ffi libffi) add_python_extension(_ctypes SOURCES ${ctypes_COMMON_SOURCES} INCLUDEDIRS ${FFI_INCLUDE_DIR} LIBRARIES ${FFI_LIBRARY} ) endif() — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#257?email_source=notifications&email_token=AABVPIZZAPKRDG3GI65WU3DQDSN4TA5CNFSM4IJ2YER2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD35C54Y#issuecomment-519712499>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABVPI422C2QFH2AQMWO7CDQDSN4TANCNFSM4IJ2YERQ> .

dand-oss · 2019-08-20T12:21:16Z

Wow, you guys have done a lot!

Anything you need me to add here?

Since python/cpython@f40d4ddff (Closes python/cpython#27979: Remove bundled copy of libffi), external version of libffi is required on linux.

jcfr · 2019-08-20T19:35:06Z

reverting libffi changes wasn't intentional, so please unrevert as applicable...

@dbrnz Thanks for the review, this last commit should do it.

jcfr · 2019-08-20T19:36:26Z

Wow, you guys have done a lot!

Thanks 😄

Anything you need me to add here?

@dand-oss doing some local testing of the branch would be helpful.

dand-oss · 2019-08-28T06:59:38Z

@dand-oss doing some local testing of the branch would be helpful.

Here is how I "checked out a local branch"
for those contributors who like me are "less than git wizards"
its not difficult!

I had a forked repo of python-cmake-buildsystem cloned to my local
with a remote and a branch named "upstream"
I regular rebased my "master" branch upon

# get the remote branch as a local branch
git remote add jcfr git@github.com:jcfr/python-cmake-buildsystem.git
git fetch jcfr
git checkout -b support-build-37-and-38-rebased jcfr/support-build-37-and-38-rebased

# get support-build-37-and-38-rebased
git rebase -i upstream

# get my master branch on top of the trial branch
# - custom cmake, ability to build debug, my own scripts to set up venv after build
git checkout master
git rebase -i support-build-37-and-38-rebased

Then I built on windows and Linux, and all was good!

jcfr · 2019-09-24T19:24:52Z

Then I built on windows and Linux, and all was good!

Great. Thanks for taking the time to test 🙏

jamesobutler · 2019-10-27T13:27:29Z

Based on the title of this PR “ Support build 37 and 38 rebased“, is this PR waiting on Python 3.8.0 support to be added? It doesn’t currently appear to be in this PR.

dand-oss · 2019-10-31T04:40:58Z

Python 3.8 will not build - requires more work

CMake Error at cmake/libpython/CMakeLists.txt:450 (add_executable):
Cannot find source file:

build-64/Python-3.8.0/Parser/bitset.c

Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx

CMake Error at cmake/libpython/CMakeLists.txt:477 (add_library):
Cannot find source file:

build-64/Python-3.8.0/Modules/zipimport.c

Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Call Stack (most recent call first):
cmake/libpython/CMakeLists.txt:513 (add_libpython)

CMake Error at cmake/libpython/CMakeLists.txt:450 (add_executable):
No SOURCES given to target: pgen

CMake Error at cmake/libpython/CMakeLists.txt:477 (add_library):
No SOURCES given to target: libpython-shared
Call Stack (most recent call first):
cmake/libpython/CMakeLists.txt:513 (add_libpython)

CMake Error at cmake/libpython/CMakeLists.txt:406 (add_executable):
No SOURCES given to target: _freeze_importlib

jamesobutler · 2019-11-10T00:30:34Z

Testing Python 3.7 support

I'm having trouble generating the solution for the default configuration in this PR for 3.7.4.

CMake Error at cmake/libpython/CMakeLists.txt:493 (add_library):
  Cannot find source file:

    C:/D/Python-3.7.4/Modules/zlib/adler32.c

This appears to be because zlib (only used for Windows) was removed from the cpython repository to be in cpython-source-deps as specified in python/cpython@d01db1c starting with python 3.7.0.

Testing Python 3.8 support

Regarding failures mentioned by @dand-oss:

CMake Error at cmake/libpython/CMakeLists.txt:450 (add_executable):
Cannot find source file:

build-64/Python-3.8.0/Parser/bitset.c

CMake Error at cmake/libpython/CMakeLists.txt:450 (add_executable):
No SOURCES given to target: pgen

Bitset.c and pgen were removed in python 3.8.0 as part of python/cpython@1f24a71.

CMake Error at cmake/libpython/CMakeLists.txt:477 (add_library):
Cannot find source file:

build-64/Python-3.8.0/Modules/zipimport.c

Zipimport was rewritten in pure python for 3.8.0. It is now at Lib/zipimport.py. See python/cpython@79d1c2e

dand-oss · 2020-01-29T20:11:09Z

Status

I am using cmake-buildystem with Python 3.7.6 on Windows and Linux

python 3.8 will take significant work - much has changed...

I put a few hours in on 3.8, worked around some problems, but more problems remain.

gongminmin · 2020-02-19T06:46:40Z

I have changes to adapt 3.7 and 3.8. It's not a general enough change, only works for my scenario. Feel free to pick useful lines from them.

3.7: gongminmin@7d97c1d
3.8: gongminmin@d4818e7

jcfr · 2022-01-09T00:14:57Z

Closing. This pull-request is superseded by #289 and #292

jcfr and others added 16 commits August 6, 2019 09:24

cmake: Support extracting version from patchlevel for alpha/beta/rc r…

051066d

…eleases

Fix extraction of version info introducing "PythonExtractVersionInfo"…

aebb1d9

… module

ci: Update appveyor/circleci/travis to test python version 3.7.0b2

8d3e717

cmake: Fix generation of downloading for pre-release python versions

b44392d

Add checksums for Python 3.5.6, 3.6.8, 3.6.9. Set default to 3.6.9

3cf08e5

3.7.x: Add checksums for Python 3.7.0 to 3.7.4

10b87c3

cmake: Add initial support for building Python 3.7.x

650d8e6

3.7.x: Add support for building new modules

0847bd0

3.7.x: Check for and set RTLD_* macros.

22d1892

cmake: The name of the import initialisation function is version depe…

12e9964

…ndent.

ci: Updates travis and CircleCI

8f57779

Add support for building _uuid module

e8749de

add patches for 3.7

f1b0368

cmake: Set default python version to 3.7.4

62831d0

_PYTHONFRAMEWORK="" to extensions

43c2428

On Linux, uses external libffi for python >= 3.7

facfdcb

Since python/cpython@f40d4ddff (Closes python/cpython#27979: Remove bundled copy of libffi), external version of libffi is required on linux.

jcfr force-pushed the support-build-37-and-38-rebased branch from 3554d50 to facfdcb Compare August 20, 2019 19:34

jcfr mentioned this pull request Aug 20, 2019

Updates to build Python 3.7 #256

Closed

jcfr added the Type: Enhancement Improvement to functionality label Aug 20, 2019

dbrnz mentioned this pull request Oct 14, 2019

Python for OpenCOR opencor/opencor#2160

Closed

dand-oss mentioned this pull request Jan 29, 2020

Upgrade to Python 3.7.6 #264

Closed

jcfr closed this Jan 9, 2022

jcfr deleted the support-build-37-and-38-rebased branch January 19, 2022 08:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support build 37 and 38 rebased #257

Support build 37 and 38 rebased #257

Uh oh!

jcfr commented Aug 6, 2019

Uh oh!

jcfr commented Aug 6, 2019

Uh oh!

dbrnz commented Aug 7, 2019

Uh oh!

dbrnz commented Aug 8, 2019

Uh oh!

jcfr commented Aug 9, 2019 via email

Uh oh!

dand-oss commented Aug 20, 2019

Uh oh!

jcfr commented Aug 20, 2019

Uh oh!

jcfr commented Aug 20, 2019

Uh oh!

dand-oss commented Aug 28, 2019 •

edited

Loading

Uh oh!

jcfr commented Sep 24, 2019

Uh oh!

jamesobutler commented Oct 27, 2019

Uh oh!

dand-oss commented Oct 31, 2019

Uh oh!

jamesobutler commented Nov 10, 2019 •

edited

Loading

Uh oh!

dand-oss commented Jan 29, 2020

Uh oh!

gongminmin commented Feb 19, 2020

Uh oh!

jcfr commented Jan 9, 2022

Uh oh!

Uh oh!

Support build 37 and 38 rebased #257

Support build 37 and 38 rebased #257

Uh oh!

Conversation

jcfr commented Aug 6, 2019

Uh oh!

jcfr commented Aug 6, 2019

Uh oh!

dbrnz commented Aug 7, 2019

Uh oh!

dbrnz commented Aug 8, 2019

Uh oh!

jcfr commented Aug 9, 2019 via email

Uh oh!

dand-oss commented Aug 20, 2019

Uh oh!

jcfr commented Aug 20, 2019

Uh oh!

jcfr commented Aug 20, 2019

Uh oh!

dand-oss commented Aug 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcfr commented Sep 24, 2019

Uh oh!

jamesobutler commented Oct 27, 2019

Uh oh!

dand-oss commented Oct 31, 2019

Uh oh!

jamesobutler commented Nov 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing Python 3.7 support

Testing Python 3.8 support

Uh oh!

dand-oss commented Jan 29, 2020

Uh oh!

gongminmin commented Feb 19, 2020

Uh oh!

jcfr commented Jan 9, 2022

Uh oh!

Uh oh!

dand-oss commented Aug 28, 2019 •

edited

Loading

jamesobutler commented Nov 10, 2019 •

edited

Loading