Skip to content

[Inductor][CPP] Remove redundant Buffers after Grouped GEMM Fusion #143904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

leslie-fang-intel
Copy link
Collaborator

@leslie-fang-intel leslie-fang-intel commented Dec 27, 2024

Stack from ghstack (oldest at bottom):

Summary
In this PR, we remove the extra kernel arguments and the extra buffers allocation when any MultiOutput Buffer is consumed by an out-template epilogue. If any MultiOutput Buffer is consumed by an out-template epilogue, the Grouped GEMM Template should bypass storing it in the MultiOutput Buffer and instead write it directly to the corresponding out-template epilogue.

Remove extra kernel arguments
For the case listed above, a MultiOutput Buffer shouldn't exist in the Kernel's args if it's consumed by an out-template epilogue. We mark this MultiOutput Buffer as REMOVED for this case.

Remove the extra buffers allocation
For the case listed above, a MultiOutput Buffer shouldn't be allocated. We introduce the outputs_removed attribute in the CppTemplateBuffer. This attribute tracks MultiOutput Buffers that are directly used by out-template epilogues. During code generation, if a MultiOutput Buffer is listed in outputs_removed, its buffer allocation line is omitted to prevent unnecessary memory usage.

Test Plan

python -u -m pytest -s -v test/inductor/test_cpu_select_algorithm.py -k test_grouped_linear_epilogue

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Dec 27, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/143904

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 71d497a with merge base 744a303 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]
leslie-fang-intel added a commit that referenced this pull request Dec 27, 2024
[ghstack-poisoned]
leslie-fang-intel added a commit that referenced this pull request Dec 30, 2024
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
@leslie-fang-intel leslie-fang-intel marked this pull request as ready for review December 31, 2024 05:57
@leslie-fang-intel leslie-fang-intel changed the title [Inductor][CPP] Remove redundant Buffers after Group GEMM Fusion [Inductor][CPP] Remove redundant Buffers after Grouped GEMM Fusion Dec 31, 2024
[ghstack-poisoned]
leslie-fang-intel added a commit that referenced this pull request Jan 2, 2025
@github-actions github-actions bot deleted the gh/leslie-fang-intel/176/head branch February 3, 2025 02:03
sunjiweiswift pushed a commit to sunjiweiswift/pytorch that referenced this pull request Jul 4, 2025
sunjiweiswift pushed a commit to sunjiweiswift/pytorch that referenced this pull request Aug 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants