Skip to content

[pyos][question] Handling of graphBLAS API version? #424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sneakers-the-rat opened this issue Apr 9, 2023 · 3 comments · Fixed by #441
Closed

[pyos][question] Handling of graphBLAS API version? #424

sneakers-the-rat opened this issue Apr 9, 2023 · 3 comments · Fixed by #441
Labels
discussion Discussing a topic with no specific actions yet documentation Improve or add to documentation

Comments

@sneakers-the-rat
Copy link

sneakers-the-rat commented Apr 9, 2023

Trying to understand the implementation of the API wrapper, and I'm wondering how the version of the graphBLAS API is handled? It looks like the API moves slowly, but it does move ( eg https://graphblas.org/ says v1.3.0 on September 25, 2019 and v2.0.0 on November 15, 2021 ).

I ask because it looks like the mappings are relatively manual - a lot of hardcoded names, etc. - rather than something where you're programmatically generating bindings directly from the API. Having just started taking a look at the implementation I am assuming there's a good reason for that, so completely fine, but is there some place where you specify that "this version of python-graphblas is compatible with x version of graphblas API?"

I see that a version of suitesparse-graphblas is pinned, so presumably that also pins some specific version of the API?

In any case, seems like something that would be worth having in the docs if not implemented in the code - there's a statement on python version here: https://python-graphblas.readthedocs.io/en/stable/getting_started/faq.html#what-is-the-version-support-policy but I don't find one for graphBLAS API version

edit: part of pyOpenSci/software-submission#81

@eriknw
Copy link
Member

eriknw commented Apr 10, 2023

is there some place where you specify that "this version of python-graphblas is compatible with x version of graphblas API?"

Not yet, but that is a good point: we should.

We don't have a strict version policy for how many or how long we'll support any given GraphBLAS C API version, because it's hard to know how future versions will change and how compatible they will be. When new versions come out, we would like to support them. When this happens, I suppose we ought to have a discussion with implementers and users to assess their needs. So, I guess our policy is "we'll talk to stakeholders when this comes up". Version 2.1 is expected to be released this year, and it should be largely compatible, so we can probably support both 2.0 and 2.1.

I see that a version of suitesparse-graphblas is pinned, so presumably that also pins some specific version of the API?

Yeah, right now SuiteSparse:GraphBLAS is the main implementation, and it implements version 2.0.0 (and it's own extensions). We plan to support other implementations, and we expect each implementation to have differences and their own extensions.

SuiteSparse:GraphBLAS has an extension API that lets you set configuration gb.ss.config and see information gb.ss.about. For example, to get the C API version: gb.ss.about["api_version"] (and I know these aren't super-well documented). This type of query is expected to be part of the GraphBLAS C API 2.1 spec.

@sneakers-the-rat
Copy link
Author

gotcha. I suppose this relates to my questions i'm having rn as i try and read the implementation of the wrapping, wondering how y'all will support multiple backends, as it seems like calls to suitesparse are mostly concentrated in the top level __init__ module, but also I see some special casing done in some utils functions and other parts of the package. I think probably that will eventually need to be split out into a specific backend handling system, which should also help sort out some of the more confusing parts in how the backend is initialized and the calls are constructed, but that seems like a "dev roadmap" kind of question to me

@eriknw
Copy link
Member

eriknw commented Apr 10, 2023

wondering how y'all will support multiple backends [...] seems like a "dev roadmap" kind of question to me

Yup, supporting multiple backends is definitely a roadmap item that will require some work. If another implementation is in C that is wrapped by cffi, then the current approach of having the lib and ffi from the cffi wrapping should "just work" for most things. If, however, another implementation uses something such as MLIR or numba, then we'll need to create a better abstraction layer. I expect/hope this will mostly be grunt work and not particularly complicated. We've already decoupled python-graphblas from relying too heavily on cffi compared to earlier.

As the team has discussed, support for multiple backends will likely come in phases. In one phase, we can support multiple backends, but only one backend at a time (kind of like now with "suitesparse" and "suitesparse-vanilla" backends). In another phase, we'd like to allow use of multiple backends simultaneously in the same Python process. We have ideas of what this might look like, but it's still largely TBD. (cygraphblas was a PoC from years ago trying one (limited) way to do this, but I think using Cython would make the code even more difficult to develop and understand).

The "suitesparse-vanilla" backend is not quite fully vanilla yet, but it's a long way there and has helped us from being too dependent on SuiteSparse:GraphBLAS implementation. I think a refactor of the operators will be in order to better support "suitesparse-vanilla" and other backends. Hopefully the refactor will make operators more comprehendible (such as expressible in JSON) and more easy to document.

We're inching our way there with other changes such as making numba (and even suitesparse-graphblas) optional.

I ask because it looks like the mappings are relatively manual - a lot of hardcoded names, etc.

I'm curious what alternative you have in mind.

@eriknw eriknw added documentation Improve or add to documentation discussion Discussing a topic with no specific actions yet labels Apr 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discussing a topic with no specific actions yet documentation Improve or add to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants