feat: Remove pyarrow and bqstorage as dependencies #847
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Temporarily labeling with owlbot:ignore. Once we have worked out exactly how things should look, I will go back and edit the owlbot files and ensure that synthtool can properly create the noxfiles as desired. |
tswast
reviewed
Mar 17, 2023
tswast
reviewed
Mar 21, 2023
…on-bigquery-sqlalchemy into remove-pyarrow-bqstorage
|
@tswast I got owlbot to work and all the tests are passing. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Removes
pyarrowandbqstorageas regular dependencies and moves them to the extras category where they can be installed, as needed (i.e. when the use ofpyarrowand/orbqstoragemay provide performance benefits).As noted in Issue #743:
pyarrowis an optional dependency ofgoogle-cloud-bigquery, but it's made mandatory bypython-bigquery-sqlalchemy.Because
pyarrowis quite large on disk — 100 MB on x86_64 Linux — installing it when it is not used is inefficient and poses a burden on some installations in space constrained environments.This is not deemed a breaking change in that no existing functionality is being deprecated and existing users still have the ability to install
bqstorage/pyarrowas needed.NOTE: Users are likely to benefit from installing
bqstoragewhen accessing query results in excess of 100 Mb ... these users may experience a speed increase up to 10x.Fixes #743 🦕
IF this PR is merged, we should be able to close the following adjacent/associated PR:
Closes #470