-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
DOC: Improve lookup documentation #61471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DOC: Improve lookup documentation #61471
Conversation
Add pd_lookup_het() and pd_lookup_hom()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! No strong opposition to having both functions, but the performance gain of the _het
version does not seem significant to me.
Addressed your concerns! If you have time for a review.
…On Wed, May 21, 2025, 4:33 PM Richard Shadrach ***@***.***> wrote:
***@***.**** requested changes on this pull request.
No strong opposition to having both functions, but the performance gain of
the _het version does not seem significant to me.
------------------------------
In doc/source/user_guide/indexing.rst
<#61471 (comment)>:
>
- df = pd.DataFrame({'col': ["A", "A", "B", "B"],
- 'A': [80, 23, np.nan, 22],
- 'B': [80, 55, 76, 67]})
- df
- idx, cols = pd.factorize(df['col'])
- df.reindex(cols, axis=1).to_numpy()[np.arange(len(df)), idx]
+ def pd_lookup_hom(df, row_labels, col_labels):
+ rows = df.index.get_indexer(row_labels)
Can you add df = df.loc[:, sorted(set(col_labels))] here.
------------------------------
In doc/source/user_guide/indexing.rst
<#61471 (comment)>:
> +
+.. code-block:: python
+
+ def pd_lookup_het(df, row_labels, col_labels):
+ rows = df.index.get_indexer(row_labels)
+ cols = df.columns.get_indexer(col_labels)
+ sub = df.take(np.unique(cols), axis=1)
+ sub = sub.take(np.unique(rows), axis=0)
+ rows = sub.index.get_indexer(row_labels)
+ values = sub.melt()["value"]
+ cols = sub.columns.get_indexer(col_labels)
+ flat_index = rows + cols * len(sub)
+ result = values[flat_index]
+ return result
+
+For homogeneous column types, it is fastest to skip column subsetting and go directly to numpy:
Nit: NumPy
------------------------------
In doc/source/user_guide/indexing.rst
<#61471 (comment)>:
>
-.. ipython:: python
+For heterogeneous column types, we subset columns to avoid unnecessary numpy conversions:
NumPy again.
—
Reply to this email directly, view it on GitHub
<#61471 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFZOHK43SLWEMO55YCEVV327TPLNAVCNFSM6AAAAAB5TRZE7WVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDQNJZGAZTKOJZGA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Thanks @stevenae |
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.Follows from #61185
Examples available at https://colab.research.google.com/drive/1MGWX6JVJL5yHyK7BeEBPQAW4tLM3TZL9#scrollTo=DjWfk4i1SiOY