Skip to content

chore: internal version of resample. #1004

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Oct 1, 2024
Merged

chore: internal version of resample. #1004

merged 17 commits into from
Oct 1, 2024

Conversation

Genesis929
Copy link
Collaborator

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. samples Issues that are directly related to samples. labels Sep 20, 2024
@Genesis929 Genesis929 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 20, 2024
@bigframes-bot bigframes-bot removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 20, 2024
@Genesis929 Genesis929 added the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 20, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 20, 2024
@Genesis929 Genesis929 added the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 23, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 23, 2024
@Genesis929 Genesis929 marked this pull request as ready for review September 23, 2024 22:07
@Genesis929 Genesis929 requested review from a team as code owners September 23, 2024 22:07
@Genesis929 Genesis929 requested a review from tswast September 23, 2024 22:07
col_id = self.index.resolve_level(level)[0]
# Reset index to make the resampling level a column, then drop all other index columns.
# This simplifies processing by focusing solely on the column required for resampling.
block = self.reset_index(drop=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might require a total ordering in BigFrames to work. Could you run some tests with unordered mode + a timestamp index that has non-unique values?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test for unordered mode and with duplicate values, seems runs fine. I don't think the order matters here, in the end we are not keeping this order.

@Genesis929 Genesis929 requested a review from tswast September 26, 2024 21:59
# Validate and resolve the index or column to use for grouping
if on is None:
if len(self.index_columns) == 0:
raise TypeError("Index type not valid. Expected Datetime Type.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you double check the error message?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Args:
rule (str):
The offset string representing target conversion.
on (str, default None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc arguments isn't match the function definition. Could you double check?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

'epoch': origin is 1970-01-01
'start': origin is the first value of the timeseries
'start_day': origin is the first day at midnight of the timeseries
Returns:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add "Examples" too? Though it's an internal API.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

elif origin == "start":
return y.cast(ibis_dtypes.Timestamp(timezone="UTC")).cast(ibis_dtypes.int64)
else:
raise ValueError(f"Origin {origin} not supported")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add some tests to cover this line of codes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two xfail tests for on and origin, but this line of code should be unreachable.

@Genesis929 Genesis929 enabled auto-merge (squash) October 1, 2024 09:02
@Genesis929 Genesis929 merged commit a5ad033 into main Oct 1, 2024
20 of 23 checks passed
@Genesis929 Genesis929 deleted the resample_inter_huanc branch October 1, 2024 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. samples Issues that are directly related to samples. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants