[inductor] consolidate common GEMM triton param retrieval #158015

coconutruben · 2025-07-10T08:42:30Z

Stack from ghstack (oldest at bottom):

# Why

Make loop iteration simpler
Have a common spot where to make modifications that affect
all the GEMM Triton templates, avoiding missed spots

# What

pull out commong logic of taking the BaseConfig objects
and turning them into kwargs to feed into maybe_append_choice
for Triton GEMM templates

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

Differential Revision: D78081314

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates [ghstack-poisoned]

pytorch-bot · 2025-07-10T08:42:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158015

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 64 Pending, 9 Unrelated Failures

As of commit 7db9acc with merge base 2523e58 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / linux-jammy-py3.9-gcc11 / test (default, 5, 5, linux.2xlarge) (gh) (detected as infra flaky with no log or failing log classifier)
trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 1, linux.rocm.gpu.4) (gh) (similar failure)
distributed/test_c10d_nccl.py::DistributedDataParallelTest::test_grad_layout_1devicemodule_1replicaperprocess
xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 1, 6, linux.idc.xpu) (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 3, 6, linux.idc.xpu) (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 4, 6, linux.idc.xpu) (gh) (disabled by #159331)
export/test_torchbind.py::TestCompileTorchbind::test_compile_obj_torchbind_op_with_autocast_device_cuda_backend_aot_eager
xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 6, 6, linux.idc.xpu) (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

inductor / cuda12.8-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
Process completed with exit code 1.
inductor / cuda12.8-py3.10-gcc9-sm86 / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
Process completed with exit code 1.

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / linux-jammy-py3_9-clang9-xla / test (xla, 1, 1, linux.12xlarge, unstable) (gh) (#158876)
sccache: error: couldn't connect to server

This comment was automatically generated by Dr. CI and updates every 15 minutes.

coconutruben · 2025-07-10T08:50:47Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

coconutruben · 2025-07-10T18:50:22Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

coconutruben · 2025-07-10T19:39:09Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-10T19:49:44Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

torch/_inductor/kernel/bmm.py

torch/_inductor/kernel/mm_common.py

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-11T00:21:49Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-11T00:37:43Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-11T00:53:30Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-11T02:15:55Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

pytorch-bot · 2025-07-25T00:13:59Z

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: the following arguments are required: -m/--message, -c/--classification

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

henrylhtsang · 2025-07-25T00:14:23Z

@pytorchbot revert -m "breaking tests" --no-signal

pytorch-bot · 2025-07-25T00:14:26Z

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: the following arguments are required: -c/--classification

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

henrylhtsang · 2025-07-25T00:14:38Z

@pytorchbot revert -m "breaking tests" -c ignoredsignal

pytorchmergebot · 2025-07-25T00:16:33Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

…58015)" This reverts commit 9faef3d. Reverted #158015 on behalf of https://github.com/henrylhtsang due to breaking tests ([comment](#158015 (comment)))

pytorchmergebot · 2025-07-25T00:16:55Z

@coconutruben your PR has been successfully reverted.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-25T08:43:30Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-25T16:40:21Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-28T09:26:01Z

@coconutruben has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

coconutruben · 2025-07-29T12:19:58Z

@pytorchbot merge

pytorchmergebot · 2025-07-29T12:21:56Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

coconutruben · 2025-07-29T12:43:26Z

@pytorchbot merge -i

pytorchmergebot · 2025-07-29T12:43:47Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

pytorchmergebot · 2025-07-29T12:45:32Z

Merge started

Your change will be merged while ignoring the following 6 checks: pull / linux-jammy-py3_9-clang9-xla / test (xla, 1, 1, linux.12xlarge, unstable), trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 1, linux.rocm.gpu.4), inductor / cuda12.8-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu), inductor / cuda12.8-py3.10-gcc9-sm86 / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu), xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 6, 6, linux.idc.xpu), xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 3, 6, linux.idc.xpu)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2025-07-29T13:23:02Z

Merge failed

Reason: 1 jobs have failed, first few of them are: xpu / linux-jammy-xpu-2025.1-py3.9 / test (default, 4, 6, linux.idc.xpu)

Details for Dev Infra team

Raised by workflow job

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) [ghstack-poisoned]

coconutruben · 2025-07-29T17:40:45Z

landing in #159383 instead

\# Why - Make loop iteration simpler - Have a common spot where to make modifications that affect all the GEMM Triton templates, avoiding missed spots \# What - pull out commong logic of taking the BaseConfig objects and turning them into kwargs to feed into maybe_append_choice for Triton GEMM templates Differential Revision: [D78081314](https://our.internmc.facebook.com/intern/diff/D78081314) Pull Request resolved: #158015 Approved by: https://github.com/PaulZhang12, https://github.com/jansel

…58015)" This reverts commit 9faef3d. Reverted #158015 on behalf of https://github.com/henrylhtsang due to breaking tests ([comment](#158015 (comment)))

coconutruben mentioned this pull request Jul 10, 2025

[inductor] initial triton static config lookup table #157699

Open

pytorch-bot bot added ciflow/inductor module: inductor labels Jul 10, 2025

coconutruben added the topic: not user facing topic category label Jul 10, 2025

coconutruben requested review from jansel and PaulZhang12 July 10, 2025 08:45

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 10, 2025

PaulZhang12 approved these changes Jul 10, 2025

View reviewed changes

jansel requested changes Jul 10, 2025

View reviewed changes

coconutruben mentioned this pull request Jul 11, 2025

[inductor] add template hashing for template lookup table #158091

Open

coconutruben added 2 commits July 21, 2025 15:37

coconutruben added the ciflow/xpu Run XPU CI tasks label Jul 22, 2025

coconutruben added 2 commits July 21, 2025 23:06

pytorchmergebot added Reverted ci-no-td Do not run TD on this PR labels Jul 25, 2025

pytorchmergebot reopened this Jul 25, 2025

pytorchmergebot added the merging label Jul 29, 2025

pytorchmergebot removed the merging label Jul 29, 2025

coconutruben mentioned this pull request Jul 29, 2025

[inductor] consolidate common GEMM triton param retrieval #159383

Closed

coconutruben closed this Jul 29, 2025

[inductor] consolidate common GEMM triton param retrieval #158015

[inductor] consolidate common GEMM triton param retrieval #158015

Uh oh!

Conversation

coconutruben commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158015

⏳ 64 Pending, 9 Unrelated Failures

Uh oh!

coconutruben commented Jul 10, 2025

Uh oh!

coconutruben commented Jul 10, 2025

Uh oh!

coconutruben commented Jul 10, 2025

Uh oh!

coconutruben commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coconutruben commented Jul 11, 2025

Uh oh!

coconutruben commented Jul 11, 2025

Uh oh!

coconutruben commented Jul 11, 2025

Uh oh!

coconutruben commented Jul 11, 2025

Uh oh!

pytorch-bot bot commented Jul 25, 2025

Uh oh!

henrylhtsang commented Jul 25, 2025

Uh oh!

pytorch-bot bot commented Jul 25, 2025

Uh oh!

henrylhtsang commented Jul 25, 2025

Uh oh!

pytorchmergebot commented Jul 25, 2025

Uh oh!

pytorchmergebot commented Jul 25, 2025

Uh oh!

coconutruben commented Jul 25, 2025

Uh oh!

coconutruben commented Jul 25, 2025

Uh oh!

coconutruben commented Jul 28, 2025

Uh oh!

coconutruben commented Jul 29, 2025

Uh oh!

pytorchmergebot commented Jul 29, 2025

Merge started

Uh oh!

coconutruben commented Jul 29, 2025

Uh oh!

pytorchmergebot commented Jul 29, 2025

Uh oh!

pytorchmergebot commented Jul 29, 2025

Merge started

Uh oh!

pytorchmergebot commented Jul 29, 2025

Merge failed

Uh oh!

coconutruben commented Jul 29, 2025

Uh oh!

Uh oh!

coconutruben commented Jul 10, 2025 •

edited

Loading

pytorch-bot bot commented Jul 10, 2025 •

edited

Loading