diff --git a/doc/source/f2py/index.rst b/doc/source/f2py/index.rst index 913eacbb131b..dedfe3f644dd 100644 --- a/doc/source/f2py/index.rst +++ b/doc/source/f2py/index.rst @@ -34,9 +34,15 @@ replace all calls to ``f2py`` mentioned in this guide with the longer version. .. toctree:: :maxdepth: 3 + f2py.getting-started f2py-user f2py-reference - + usage + python-usage + signature-file + buildtools/index + advanced + windows/index .. _Python: https://www.python.org/ .. _NumPy: https://www.numpy.org/ diff --git a/doc/source/f2py/windows/conda.rst b/doc/source/f2py/windows/conda.rst new file mode 100644 index 000000000000..b16402bbcf2f --- /dev/null +++ b/doc/source/f2py/windows/conda.rst @@ -0,0 +1,34 @@ +.. _f2py-win-conda: + +========================= +F2PY and Conda on Windows +========================= + +As a convienience measure, we will additionally assume the +existence of ``scoop``, which can be used to install tools without +administrative access. + +.. code-block:: powershell + + Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh') + +Now we will setup a ``conda`` environment. + +.. code-block:: powershell + + scoop install miniconda3 + # For conda activate / deactivate in powershell + conda install -n root -c pscondaenvs pscondaenvs + Powershell -c Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser + conda init powershell + # Open a new shell for the rest + +``conda`` pulls packages from ``msys2``, however, the UX is sufficiently different enough to warrant a separate discussion. + +.. warning:: + + As of 30-01-2022, the `MSYS2 binaries`_ shipped with ``conda`` are **outdated** and this approach is **not preferred**. + + + +.. _MSYS2 binaries: https://github.com/conda-forge/conda-forge.github.io/issues/1044 diff --git a/doc/source/f2py/windows/index.rst b/doc/source/f2py/windows/index.rst new file mode 100644 index 000000000000..aee96cb13c6d --- /dev/null +++ b/doc/source/f2py/windows/index.rst @@ -0,0 +1,208 @@ +.. _f2py-windows: + +================= +F2PY and Windows +================= + +.. warning:: + + F2PY support for Windows is not at par with Linux support, and + OS specific flags can be seen via ``python -m numpy.f2py`` + +Broadly speaking, there are two issues working with F2PY on Windows: + +- the lack of actively developed FOSS Fortran compilers, and, +- the linking issues related to the C runtime library for building Python-C extensions. + +The focus of this section is to establish a guideline for developing and +extending Fortran modules for Python natively, via F2PY on Windows. + +Overview +======== +From a user perspective, the most UNIX compatible Windows +development environment is through emulation, either via the Windows Subsystem +on Linux, or facilitated by Docker. In a similar vein, traditional +virtualization methods like VirtualBox are also reasonable methods to develop +UNIX tools on Windows. + +Native Windows support is typically stunted beyond the usage of commercial compilers. +However, as of 2022, most commercial compilers have free plans which are sufficient for +general use. Additionally, the Fortran language features supported by ``f2py`` +(partial coverage of Fortran 2003), means that newer toolchains are often not +required. Briefly, then, for an end user, in order of use: + +Classic Intel Compilers (commercial) + These are maintained actively, though licensing restrictions may apply as + further detailed in :ref:`f2py-win-intel`. + + Suitable for general use for those building native Windows programs by + building off of MSVC. + +MSYS2 (FOSS) + In conjunction with the ``mingw-w64`` project, ``gfortran`` and ``gcc`` + toolchains can be used to natively build Windows programs. + +Windows Subsystem for Linux + Assuming the usage of ``gfortran``, this can be used for cross-compiling + Windows applications, but is significantly more complicated. + +Conda + Windows support for compilers in ``conda`` is facilitated by pulling MSYS2 + binaries, however these `are outdated`_, and therefore not recommended (as of 30-01-2022). + +PGI Compilers (commercial) + Unmaintained but sufficient if an existing license is present. Works + natively, but has been superseded by the Nvidia HPC SDK, with no `native + Windows support`_. + +Cygwin (FOSS) + Can also be used for ``gfortran``. Howeve, the POSIX API compatibility layer provided by + Cygwin is meant to compile UNIX software on Windows, instead of building + native Windows programs. This means cross compilation is required. + +The compilation suites described so far are compatible with the `now +deprecated`_ ``np.distutils`` build backend which is exposed by the F2PY CLI. +Additional build system usage (``meson``, ``cmake``) as described in +:ref:`f2py-bldsys` allows for a more flexible set of compiler +backends including: + +Intel oneAPI + The newer Intel compilers (``ifx``, ``icx``) are based on LLVM and can be + used for native compilation. Licensing requirements can be onerous. + +Classic Flang (FOSS) + The backbone of the PGI compilers were cannibalized to form the "classic" or + `legacy version of Flang`_. This may be compiled from source and used + natively. `LLVM Flang`_ does not support Windows yet (30-01-2022). + +LFortran (FOSS) + One of two LLVM based compilers. Not all of F2PY supported Fortran can be + compiled yet (30-01-2022) but uses MSVC for native linking. + + +Baseline +======== + +For this document we will asume the following basic tools: + +- The IDE being considered is the community supported `Microsoft Visual Studio Code`_ +- The terminal being used is the `Windows Terminal`_ +- The shell environment is assumed to be `Powershell 7.x`_ +- Python 3.10 from `the Microsoft Store`_ and this can be tested with + ``Get-Command python.exe`` resolving to + ``C:\Users\$USERNAME\AppData\Local\Microsoft\WindowsApps\python.exe`` +- The Microsoft Visual C++ (MSVC) toolset + +With this baseline configuration, we will further consider a configuration +matrix as follows: + +.. _table-f2py-winsup-mat: + +.. table:: Support matrix, exe implies a Windows installer + + +----------------------+--------------------+-------------------+ + | **Fortran Compiler** | **C/C++ Compiler** | **Source** | + +======================+====================+===================+ + | Intel Fortran | MSVC / ICC | exe | + +----------------------+--------------------+-------------------+ + | GFortran | MSVC | MSYS2/exe | + +----------------------+--------------------+-------------------+ + | GFortran | GCC | WSL | + +----------------------+--------------------+-------------------+ + | Classic Flang | MSVC | Source / Conda | + +----------------------+--------------------+-------------------+ + | Anaconda GFortran | Anaconda GCC | exe | + +----------------------+--------------------+-------------------+ + +For an understanding of the key issues motivating the need for such a matrix +`Pauli Virtanen's in-depth post on wheels with Fortran for Windows`_ is an +excellent resource. An entertaining explanation of an application binary +interface (ABI) can be found in this post by `JeanHeyd Meneide`_. + +Powershell and MSVC +==================== + +MSVC is installed either via the Visual Studio Bundle or the lighter (preferred) +`Build Tools for Visual Studio`_ with the ``Desktop development with C++`` +setting. + +.. note:: + + This can take a significant amount of time as it includes a download of around + 2GB and requires a restart. + +It is possible to use the resulting environment from a `standard command +prompt`_. However, it is more pleasant to use a `developer powershell`_, +with a `profile in Windows Terminal`_. This can be achieved by adding the +following block to the ``profiles->list`` section of the JSON file used to +configure Windows Terminal (see ``Settings->Open JSON file``): + +.. code-block:: json + + { + "name": "Developer PowerShell for VS 2019", + "commandline": "powershell.exe -noe -c \"$vsPath = (Join-Path ${env:ProgramFiles(x86)} -ChildPath 'Microsoft Visual Studio\\2019\\BuildTools'); Import-Module (Join-Path $vsPath 'Common7\\Tools\\Microsoft.VisualStudio.DevShell.dll'); Enter-VsDevShell -VsInstallPath $vsPath -SkipAutomaticLocation\"", + "icon": "ms-appx:///ProfileIcons/{61c54bbd-c2c6-5271-96e7-009a87ff44bf}.png" + } + +Now, testing the compiler toolchain could look like: + +.. code-block:: powershell + + # New Windows Developer Powershell instance / tab + # or + $vsPath = (Join-Path ${env:ProgramFiles(x86)} -ChildPath 'Microsoft Visual Studio\\2019\\BuildTools'); + Import-Module (Join-Path $vsPath 'Common7\\Tools\\Microsoft.VisualStudio.DevShell.dll'); + Enter-VsDevShell -VsInstallPath $vsPath -SkipAutomaticLocation + ********************************************************************** + ** Visual Studio 2019 Developer PowerShell v16.11.9 + ** Copyright (c) 2021 Microsoft Corporation + ********************************************************************** + cd $HOME + echo "#include" > blah.cpp; echo 'int main(){printf("Hi");return 1;}' >> blah.cpp + cl blah.cpp + .\blah.exe + # Hi + rm blah.cpp + +It is also possible to check that the environment has been updated correctly +with ``$ENV:PATH``. + + +Windows Store Python Paths +========================== + +The MS Windows version of Python discussed here installs to a non-deterministic +path using a hash. This needs to be added to the ``PATH`` variable. + +.. code-block:: powershell + + $Env:Path += ";$env:LOCALAPPDATA\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\scripts" + +.. toctree:: + :maxdepth: 2 + + intel + msys2 + conda + pgi + + +.. _the Microsoft Store: https://www.microsoft.com/en-us/p/python-310/9pjpw5ldxlz5 +.. _Microsoft Visual Studio Code: https://code.visualstudio.com/Download +.. _more complete POSIX environment: https://www.cygwin.com/ +.. _This MSYS2 document: https://www.msys2.org/wiki/How-does-MSYS2-differ-from-Cygwin/ +.. _Build Tools for Visual Studio: https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019 +.. _Windows Terminal: https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk701?activetab=pivot:overviewtab +.. _Powershell 7.x: https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-windows?view=powershell-7.1 +.. _standard command prompt: https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line?view=msvc-160#developer_command_file_locations +.. _developer powershell: https://docs.microsoft.com/en-us/visualstudio/ide/reference/command-prompt-powershell?view=vs-2019 +.. _profile in Windows Terminal: https://techcommunity.microsoft.com/t5/microsoft-365-pnp-blog/add-developer-powershell-and-developer-command-prompt-for-visual/ba-p/2243078 +.. _Pauli Virtanen's in-depth post on wheels with Fortran for Windows: https://pav.iki.fi/blog/2017-10-08/pywingfortran.html#building-python-wheels-with-fortran-for-windows +.. _Nvidia HPC SDK: https://www.pgroup.com/index.html +.. _JeanHeyd Meneide: https://thephd.dev/binary-banshees-digital-demons-abi-c-c++-help-me-god-please +.. _legacy version of Flang: https://github.com/flang-compiler/flang +.. _native Windows support: https://developer.nvidia.com/nvidia-hpc-sdk-downloads#collapseFour +.. _are outdated: https://github.com/conda-forge/conda-forge.github.io/issues/1044 +.. _now deprecated: https://github.com/numpy/numpy/pull/20875 +.. _LLVM Flang: https://releases.llvm.org/11.0.0/tools/flang/docs/ReleaseNotes.html diff --git a/doc/source/f2py/windows/intel.rst b/doc/source/f2py/windows/intel.rst new file mode 100644 index 000000000000..ab0cea219e70 --- /dev/null +++ b/doc/source/f2py/windows/intel.rst @@ -0,0 +1,57 @@ +.. _f2py-win-intel: + +============================== +F2PY and Windows Intel Fortran +============================== + +As of NumPy 1.23, only the classic Intel compilers (``ifort``) are supported. + +.. note:: + + The licensing restrictions for beta software `have been relaxed`_ during + the transition to the LLVM backed ``ifx/icc`` family of compilers. + However this document does not endorse the usage of Intel in downstream + projects due to the issues pertaining to `disassembly of components and + liability`_. + + Neither the Python Intel installation nor the `Classic Intel C/C++ + Compiler` are required. + +- The `Intel Fortran Compilers`_ come in a combined installer providing both + Classic and Beta versions; these also take around a gigabyte and a half or so. + +We will consider the classic example of the generation of Fibonnaci numbers, +``fib1.f``, given by: + +.. literalinclude:: ../code/fib1.f + :language: fortran + +For ``cmd.exe`` fans, using the Intel oneAPI command prompt is the easiest approach, as +it loads the required environment for both ``ifort`` and ``msvc``. Helper batch +scripts are also provided. + +.. code-block:: bat + + # cmd.exe + "C:\Program Files (x86)\Intel\oneAPI\setvars.bat" + python -m numpy.f2py -c fib1.f -m fib1 + python -c "import fib1; import numpy as np; a=np.zeros(8); fib1.fib(a); print(a)" + +Powershell usage is a little less pleasant, and this configuration now works with MSVC as: + +.. code-block:: powershell + + # Powershell + python -m numpy.f2py -c fib1.f -m fib1 --f77exec='C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64\ifort.exe' --f90exec='C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\bin\intel64\ifort.exe' -L'C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\compiler\lib\ia32' + python -c "import fib1; import numpy as np; a=np.zeros(8); fib1.fib(a); print(a)" + # Alternatively, set environment and reload Powershell in one line + cmd.exe /k '"C:\Program Files (x86)\Intel\oneAPI\setvars.bat" && powershell' + python -m numpy.f2py -c fib1.f -m fib1 + python -c "import fib1; import numpy as np; a=np.zeros(8); fib1.fib(a); print(a)" + +Note that the actual path to your local installation of `ifort` may vary, and the command above will need to be updated accordingly. + +.. _have been relaxed: https://www.intel.com/content/www/us/en/developer/articles/release-notes/oneapi-fortran-compiler-release-notes.html +.. _disassembly of components and liability: https://software.sintel.com/content/www/us/en/develop/articles/end-user-license-agreement.html +.. _Intel Fortran Compilers: https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#inpage-nav-6-1 +.. _Classic Intel C/C++ Compiler: https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#inpage-nav-6-undefined \ No newline at end of file diff --git a/doc/source/f2py/windows/msys2.rst b/doc/source/f2py/windows/msys2.rst new file mode 100644 index 000000000000..68c4353111c9 --- /dev/null +++ b/doc/source/f2py/windows/msys2.rst @@ -0,0 +1,19 @@ +.. _f2py-win-msys2: + +=========================== +F2PY and Windows with MSYS2 +=========================== + +Follow the standard `installation instructions`_. Then, to grab the requisite Fortran compiler with ``MVSC``: + +.. code-block:: bash + + # Assuming a fresh install + pacman -Syu # Restart the terminal + pacman -Su # Update packages + # Get the toolchains + pacman -S --needed base-devel gcc-fortran + pacman -S mingw-w64-x86_64-toolchain + + +.. _`installation instructions`: https://www.msys2.org/ diff --git a/doc/source/f2py/windows/pgi.rst b/doc/source/f2py/windows/pgi.rst new file mode 100644 index 000000000000..3139d9c5b574 --- /dev/null +++ b/doc/source/f2py/windows/pgi.rst @@ -0,0 +1,28 @@ +.. _f2py-win-pgi: + +=============================== +F2PY and PGI Fortran on Windows +=============================== + +A variant of these are part of the so called "classic" Flang, however, +as classic Flang requires a custom LLVM and compilation from sources. + +.. warning:: + + Since the proprietary compilers are no longer available for + usage they are not recommended and will not be ported to the + new ``f2py`` CLI. + + + +.. note:: + + **As of November 2021** + + As of 29-01-2022, `PGI compiler toolchains`_ have been superceeded by the Nvidia + HPC SDK, with no `native Windows support`_. + +However, + +.. _PGI compiler toolchains: https://www.pgroup.com/index.html +.. _native Windows support: https://developer.nvidia.com/nvidia-hpc-sdk-downloads#collapseFour \ No newline at end of file