Skip to content

chore: update local benchmark, 10t config and kokoro project. #995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 18, 2024

Conversation

Genesis929
Copy link
Collaborator

@Genesis929 Genesis929 commented Sep 17, 2024

A few changes:

  1. Set the benchmark to run n times with -i or --iterations.
  2. Set the benchmark to run only specific folders or files with -b or --benchmark.
  3. Set local CSV output with -o or --output-csv.
  4. Set local benchmark results to be published to a specific project with the environment variable "GCLOUD_BENCH_PUBLISH_PROJECT".
  5. Set the project ID for the TPCH data read location.
  6. Updated the configuration so that TPCH will run with a 10T table.
  7. Updated the Kokoro GOOGLE_CLOUD_PROJECT configuration for benchmarking.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Sep 17, 2024
@Genesis929 Genesis929 marked this pull request as ready for review September 17, 2024 21:47
@Genesis929 Genesis929 requested review from a team as code owners September 17, 2024 21:47
@@ -13,7 +13,7 @@ env_vars: {

env_vars: {
key: "GOOGLE_CLOUD_PROJECT"
value: "bigframes-load-testing"
value: "bigframes-benchmarking"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+@shobsi, who wants to modify the GCS project for notebook tests: https://github.com/googleapis/python-bigquery-dataframes/pull/976/files

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed change for this file for now, latter will remove benchmarking for streaming then change project.

@Genesis929 Genesis929 enabled auto-merge (squash) September 18, 2024 18:02
@Genesis929 Genesis929 merged commit 7dbda84 into main Sep 18, 2024
21 of 23 checks passed
@Genesis929 Genesis929 deleted the benchmark_local_huanc branch September 18, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants