Add magics tutorial with BigQuery Storage API integration. #2087
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a notebooks tutorial, modeled after the Jupyter notebook example code for BigQuery. Use some caution when running these tests, as they run some large-ish (5 GB processed) queries and download about 500 MB worth of data. This is intentional, as the BigQuery Storage API is most useful for downloading large results.
shollyman
approved these changes
Apr 8, 2019
There was a problem hiding this comment.
I see why you'd like to convert transform simple queries to a set of projection columns and a row filter for the storage case, as it would avoid the intermediate query.
Only issue is related to the ongoing testing costs. Looks like dependencies table is ~7GB which is probably still reasonable given it can short circuit and the query looks cacheable.
plamut
pushed a commit
to plamut/python-bigquery-storage
that referenced
this pull request
Sep 2, 2020
…oogleCloudPlatform/python-docs-samples#2087) * Add magics tutorial with BigQuery Storage API integration. This is a notebooks tutorial, modeled after the Jupyter notebook example code for BigQuery. Use some caution when running these tests, as they run some large-ish (5 GB processed) queries and download about 500 MB worth of data. This is intentional, as the BigQuery Storage API is most useful for downloading large results. * Update deps. * Don't run big queries on Travis.
plamut
pushed a commit
to googleapis/python-bigquery-storage
that referenced
this pull request
Sep 10, 2020
…oogleCloudPlatform/python-docs-samples#2087) * Add magics tutorial with BigQuery Storage API integration. This is a notebooks tutorial, modeled after the Jupyter notebook example code for BigQuery. Use some caution when running these tests, as they run some large-ish (5 GB processed) queries and download about 500 MB worth of data. This is intentional, as the BigQuery Storage API is most useful for downloading large results. * Update deps. * Don't run big queries on Travis.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a notebooks tutorial, modeled after the Jupyter notebook example
code for BigQuery. Use some caution when running these tests, as they
run some large-ish (5 GB processed) queries and download about 500 MB
worth of data. This is intentional, as the BigQuery Storage API is most
useful for downloading large results.