Skip to content

Convert ft2font extension to pybind11 #28785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 18, 2024

Conversation

QuLogic
Copy link
Member

@QuLogic QuLogic commented Sep 6, 2024

PR summary

Unlike #9763, this is not a full rewrite, but more of a refactor. There may be further cleanup possible in the future, but this is somewhat large enough already.

PR checklist

@QuLogic QuLogic mentioned this pull request Sep 6, 2024
13 tasks
@tacaswell tacaswell added the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 6, 2024
@tacaswell tacaswell added this to the v3.10.0 milestone Sep 6, 2024
@github-actions github-actions bot removed the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 7, 2024
@QuLogic
Copy link
Member Author

QuLogic commented Sep 7, 2024

I do not understand the remaining stubtest failures; _tri is similar and doesn't complain about metaclasses (is it because it's private?).

I'm also not sure what to do about Glyph; we formerly did not define a constructor so that it couldn't be created from Python and so does this PR, but stubtest seems to be indicating that pybind11 created one anyway?

@ksunden
Copy link
Member

ksunden commented Sep 7, 2024

Okay, so on current main, Glyph is not accessible as a class at all in python (i.e. it AttributeErrors if you try)

The methods which return a Glyph do work, and return something that the repr says it should be at matplotlib.ft2font.Glyph, but no such class actually exists in python-land.

This PR changes that such that the Python class does exist.

As for the metaclass, my guess is that that is effectively the same problem.

My leaning would be go ahead and add a constructor, it doesn't hurt anything and may solve these issues. That or see if you can omit Glyph from the PYMODULE and it just works.

I'm not sure why stubtest was not erroring about the class not existing at runtime previously, but it wasn't.

The type hints see no __init__ as inheriting object's init. Not sure what pybind is doing, but it does seem like there is a default *args, **kwargs added.

@github-actions github-actions bot added the Documentation: build building the docs label Sep 10, 2024
@QuLogic QuLogic force-pushed the ft2font-pybind11 branch 2 times, most recently from 408b128 to 2079fc8 Compare September 10, 2024 08:22
@QuLogic
Copy link
Member Author

QuLogic commented Sep 10, 2024

For Glyph, I set the module to nullptr and unexpectedly that worked, and didn't seem to break anything.

For the metaclass, I added Buffer and that seems to have fixed it locally. But it seems __buffer__ is only part of 3.12+ with pybind11, so that fails instead.

@QuLogic QuLogic added the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 10, 2024
@github-actions github-actions bot removed the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 10, 2024
@QuLogic QuLogic added the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 10, 2024
Inline `convert_xys_to_array` and modify the arguments to take a C++
container, so we don't need a less-safe pointer, and we don't need to
copy another time over.
This allows pybind11 to generate the type hints for us, and it also
takes care of checking the list and its contents are the right type.
The former is not available on the macOS deployment target we use for
wheels. We could revert back to `PyOS_snprintf`, but C++11 contains
`snprintf`, and it seems to guarantee the same things.
And also ignore the `numpy.float64` reference. The latter seems to be
broken since Sphinx tries to auto-link type hints as `py:class`, but
it's an alias in NumPy making it a `py:attr` in their inventory.
@github-actions github-actions bot removed the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 12, 2024
@QuLogic QuLogic added the CI: Run cibuildwheel Run wheel building tests on a PR label Sep 12, 2024
@QuLogic
Copy link
Member Author

QuLogic commented Sep 12, 2024

For Glyph, I set the module to nullptr and unexpectedly that worked, and didn't seem to break anything.

Reverted that, since it causes weird results in the docs. Now Glyph has an __init__, but it raises immediately.

@@ -232,23 +215,73 @@ class FT2Font:
def set_text(
self, string: str, angle: float = ..., flags: int = ...
) -> NDArray[np.float64]: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if np.double is the correct return type?

https://numpy.org/doc/stable/reference/arrays.scalars.html#numpy.double

From reading the documentation it seems like float64 is an alias for double on some (probably most though) platforms.

(Not added in the PR, but the missing reference is...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thought, but unfortunately it doesn't fix the missing reference. It appears that Sphinx doesn't read any .pyi files (which is why most other pages don't have type hint issues), but in this case is seeing something that pybind11 is generating in the runtime itself.

@ianthomas23
Copy link
Member

I was going to say that we shouldn't need
https://github.com/QuLogic/matplotlib/blob/a0649e792797e16e5c2d84e267c9a848b9905272/src/ft2font_wrapper.cpp#L7
or
https://github.com/QuLogic/matplotlib/blob/a0649e792797e16e5c2d84e267c9a848b9905272/src/ft2font_wrapper.cpp#L958-L964
as ft2font_wrapper.cpp doesn't access numpy directly but does so via pybind11. Trying this out locally everything builds and runs OK. Then I realised that we could correspondingly remove 'numpy_dep' from
https://github.com/QuLogic/matplotlib/blob/a0649e792797e16e5c2d84e267c9a848b9905272/src/meson.build#L97
but that leads to trouble due to the eventual import of mplutils.h which does some low-level stuff even though I think we only actually need the kind codes. So I am backing out of all of this in the interests of getting this merged now and dealing with the "everything is now pybind11" cleanup that @QuLogic has mentioned elsewhere.

}

PyFT2Font *self = new PyFT2Font();
self->x = NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we'd use the C++ nullptr instead of NULL, but as there are a number of NULLs already in this file let's leave it consistent with that.

@ianthomas23
Copy link
Member

Merging as there are 2 approvals.

I intend to make time to review #27011 sometime in the next week.

@ianthomas23 ianthomas23 merged commit 9674b53 into matplotlib:main Sep 18, 2024
56 checks passed
@tacaswell
Copy link
Member

My understanding is that @QuLogic has some follow on work to finish removing our build-time numpy dependency all together.

@QuLogic QuLogic deleted the ft2font-pybind11 branch September 18, 2024 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants