-
Notifications
You must be signed in to change notification settings - Fork 49
chore: internal version of resample. #1004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
e8ec920
to
240c1e8
Compare
col_id = self.index.resolve_level(level)[0] | ||
# Reset index to make the resampling level a column, then drop all other index columns. | ||
# This simplifies processing by focusing solely on the column required for resampling. | ||
block = self.reset_index(drop=False) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might require a total ordering in BigFrames to work. Could you run some tests with unordered mode + a timestamp index that has non-unique values?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a test for unordered mode and with duplicate values, seems runs fine. I don't think the order matters here, in the end we are not keeping this order.
bigframes/core/blocks.py
Outdated
# Validate and resolve the index or column to use for grouping | ||
if on is None: | ||
if len(self.index_columns) == 0: | ||
raise TypeError("Index type not valid. Expected Datetime Type.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you double check the error message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.
bigframes/series.py
Outdated
Args: | ||
rule (str): | ||
The offset string representing target conversion. | ||
on (str, default None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The doc arguments isn't match the function definition. Could you double check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
'epoch': origin is 1970-01-01 | ||
'start': origin is the first value of the timeseries | ||
'start_day': origin is the first day at midnight of the timeseries | ||
Returns: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add "Examples" too? Though it's an internal API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
elif origin == "start": | ||
return y.cast(ibis_dtypes.Timestamp(timezone="UTC")).cast(ibis_dtypes.int64) | ||
else: | ||
raise ValueError(f"Origin {origin} not supported") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please add some tests to cover this line of codes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added two xfail tests for on and origin, but this line of code should be unreachable.
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕