-
-
Notifications
You must be signed in to change notification settings - Fork 25.8k
DOC Add a 'Cython Best Practices, Conventions and Knowledge' section #25608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: Meekail Zain <Micky774@users.noreply.github.com> Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com> Co-authored-by: Vincent M <maladiere.vincent@yahoo.fr>
@jakirkham might be interested. |
A first comment before a proper review: I think it would be nice to point towards this new section from the reading the existing code base and the performance tips for the cython developer sections. It could be a simple "for more information, see section ...". |
@ArturoAmorQ: feel free to open a pull-request on my branch if you think this can be better worded and if it is easier for you. 👍 |
Relevant notes from @da-woods people might be interested reading: https://cython-guidelines.readthedocs.io/en/latest/ |
You can cherrypick my commit here: ArturoAmorQ/scikit-learn@ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's something I would have liked to read when I joined the project, thanks for working on this. Here are some comments.
In addition in the performance page, there's this old TODO
TODO: html report, type declarations, bound checks, division by zero checks, memory alignment, direct blas calls...
I think this PR is a good opportunity to finally do it :)
In particular, I think it's worth adding a bullet point about direct calls to blas: when it's desirable, must be through the cython_blas extension... The rest is kind of already done I think
Signed-off-by: Julien Jerphanion <git@jjerphan.xyz> Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I much enjoy and appreciate the snippets of code I can just copy and paste.
I think we are interesting in merging more of this sort of this into the Cython documentation. It probably won't be all my notes - I know there's a few of my opinions (e.g. my abiding dislike of I still think duplication doesn't hurt so there's no reason not have this section in |
I tend to agree with all of your notes. While it is useful to have Cython's developers and users (e.g. you and me here) share their experience, tips and conventions about the language, I think everyone would benefit from having them unified at one canonical place (ideally Cython's docs). Do you think it possible to identify and extract non-opiniated, factual elements from both yours and those notes and integrates them in Cython's docs ? Also, were you referring to scikit-learn instead of pandas in your previous comments ? |
I'll try to make a start on that in the next few weeks (probably bit-by-bit)
Oops yes sorry! I got temporarily dislocated! |
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com> Co-authored-by: Adam Li <adam2392@gmail.com>
Co-authored-by: Jérémie du Boisberranger <jeremiedbb@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another pass, mostly regarding the rendering this time.
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>
Added the "no changelog" label, as I think this wouldn't get added. I like it and find it useful as is. As such, should we try and merge this as soon as this round of comments is addressed? It seems like work on this PR could go on for a long time because there is so much to know. However the longer it is unmerged, the longer people won't profit from what is already here. Can always do a second, third, fourth PR to keep working on this. WDYT? |
Yes, I agree. I think Jérémie is busy with the 1.2.2 release, but other maintainers can give their +1. I am waiting for this PR to get 2 approvals with potential proposals for subsequent PRs improving the section. |
I'm also in favor of merging a good enough version and improve in subsequent PRs |
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
@jeremiedbb: I think I've treated all of your comments. |
Reading the discussions in #25780, I think it will be worth adding a bullet point about including |
I agree 💯 It is better to have 4 smaller PRs and to iterate faster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let‘s iterate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. There's just this rendering issue that bothers me :/
Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
last nitpick. The extra indentation made the first sentence bold, but all other bullet points don't so let's remove it for consistency
Thanks |
Thanks everyone! I hope this will help people getting started using Cython for scikit-learn. |
|
||
* Always prefer memoryviews instead over ``cnp.ndarray`` when possible: memoryviews are lightweight. | ||
|
||
* Avoid memoryview slicing: memoryview slicing might be costly or misleading in some cases and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FWIW I have an PR that should dramatically improve this. It almost certainly won't (and shouldn't!) make it into Cython 3, but hopefully won't be long after that. Essentially, a lot of the reference counting turns out to be pretty pointless since you're usually incrementing and decrementing exactly the same object so it can be skipped completely.
So hopefully this advice can change sometime in the next year. But not yet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for mentioning this, @da-woods.
Reference Issues/PRs
What does this implement/fix? Explain your changes.
This adds a section to scikit-learn documentation regarding some Cython know-how of the team.
Any other comments?
Feel free to suggest changes, adaptations or additions.