Skip to content

MAINT: speed up hstack and vstack by eliminating list comprehension. #13697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 2, 2019
Merged

MAINT: speed up hstack and vstack by eliminating list comprehension. #13697

merged 1 commit into from
Jun 2, 2019

Conversation

mhvk
Copy link
Contributor

@mhvk mhvk commented Jun 2, 2019

While discussing optimizations in #13634, it was clear that there was a simple optimization that was missed: atleast_*d was called on every array rather than just passing it the whole tuple (causing dispatch overhead for each if nothing else). Removing that,

x = np.array([1])
xs = [x, x, x]
%timeit np.hstack(xs)
# 4.33 ->3.45 us
%timeit np.vstack(xs)
# 5.35->4.33 us

@mhvk mhvk requested review from seberg and shoyer June 2, 2019 00:23
@mhvk
Copy link
Contributor Author

mhvk commented Jun 2, 2019

@seberg - the request to you was just because you were the "suggested" reviewer and I clicked wrong.

@shoyer - since this was discussed in #13634, I thought you might do a quick review.

@mhvk mhvk mentioned this pull request Jun 2, 2019
Merged
shoyer
shoyer approved these changes Jun 2, 2019
View reviewed changes
Copy link
Member

@shoyer shoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but don't forget about dstack which is defined in numpy/core/shape_base.py.

(Actually, it would be great to move dstack here, so they're all in one place!)

@shoyer
Copy link
Member

shoyer commented Jun 2, 2019

By the way, nice find!

@mhvk mhvk force-pushed the hstack-vstack-optimization branch from f57dbec to f034278 Compare June 2, 2019 02:03
@mhvk
Copy link
Contributor Author

mhvk commented Jun 2, 2019

OK, I added dstack - but didn't move it in this PR (probably best done as a more general merge - quite topical, though, given discussion on mailing list!)

seberg
seberg approved these changes Jun 2, 2019
View reviewed changes
Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always feel free to ask me for review... In any case, was wondering for a second if len(tup) == 1 is the nicer if, but it does not matter really.

@mhvk
Copy link
Contributor Author

mhvk commented Jun 2, 2019

Yes, I even had arrs = atleast_1d(tup) if len(tup) > 1 else [atleast_1d(tup)] at some point but ended disliking the two separate invocations.

Anyway, will merge since both of you approved.

@mhvk mhvk merged commit 52ddda6 into numpy:master Jun 2, 2019
@mhvk mhvk deleted the hstack-vstack-optimization branch June 2, 2019 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seberg seberg seberg approved these changes

@shoyer shoyer shoyer approved these changes

Assignees
No one assigned
Labels
03 - Maintenance component: numpy._core
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mhvk @shoyer @seberg