Skip to content

[WIP][1/N] Port 5 _composable/fsdp distributed test cases to Intel GPU #159118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[WIP][1/N] Port 5 _composable/fsdp distributed test cases to Intel GPU #159118

wants to merge 2 commits into from
+44 −13

Conversation

zxd1997066
Copy link
Contributor

@zxd1997066 zxd1997066 commented Jul 25, 2025 •
edited
Loading

For #114850, we will port distributed tests to Intel GPU.
We could enable Intel GPU with following methods and try the best to keep the original code styles:

  • use "torch.accelerator.current_accelerator()" to determine the accelerator backend
  • enabled XPU for some test path
  • skip some test cases which Intel GPU does not support

cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @gujinghui @EikanWang @fengyuan14 @guangyey

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 25, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159118

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 8 Pending

As of commit 47762fd with merge base 70b4a88 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added oncall: distributed Add this issue/PR to distributed oncall triage queue topic: not user facing topic category labels Jul 25, 2025
@zxd1997066 zxd1997066 force-pushed the xiangdong/dist_upstream_p1 branch from a170c4c to d418054 Compare July 25, 2025 03:12
@zxd1997066 zxd1997066 changed the title [WIP][1/N] Port 4 _composable/fsdp distributed test cases to Intel GPU [WIP][1/N] Port 5 _composable/fsdp distributed test cases to Intel GPU Jul 25, 2025
@zxd1997066 zxd1997066 marked this pull request as ready for review July 28, 2025 02:37
@zxd1997066
Copy link
Contributor Author

@pytorchbot label "module: xpu"

@pytorch-bot pytorch-bot bot added the module: xpu Intel XPU related issues label Jul 28, 2025
@zxd1997066
Copy link
Contributor Author

@pytorchbot label "triaged"

@pytorch-bot pytorch-bot bot added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jul 28, 2025
@zxd1997066 zxd1997066 mentioned this pull request Jul 28, 2025
Open
guangyey
guangyey reviewed Jul 30, 2025
View reviewed changes
test/distributed/_composable/fsdp/test_fully_shard_compile.py Outdated
@@ -129,6 +129,9 @@ class TestFullyShardCompile(FSDPTest):
def skipTestForOldSm(self):
# Assumption: This test class is only run on GPU. See `HAS_GPU` check at
# the top of the class.
# XPU is not applicable in this function
if device_type == "xpu":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here should provide a whitelist mechanism, skip test iff. device_type = 'cuda' and not sm_is_or_higher_than(device, 8, 0)

guangyey
guangyey reviewed Jul 30, 2025
View reviewed changes
test/distributed/_composable/fsdp/test_fully_shard_training.py Outdated
Comment on lines 328 to 329
@unittest.skipIf(TEST_HPU, "Sleep kernel not supported for HPU")
@unittest.skipIf(TEST_XPU, "sleep kernel not supported on XPU")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@unittest.skipIf(TEST_HPU, "Sleep kernel not supported for HPU")
@unittest.skipIf(TEST_XPU, "sleep kernel not supported on XPU")
@unittest.skipIf(TEST_HPU or TEST_XPU, "Sleep kernel not supported.")

@zxd1997066 zxd1997066 force-pushed the xiangdong/dist_upstream_p1 branch from d418054 to 1b3ead9 Compare July 30, 2025 03:58
@zxd1997066 zxd1997066 force-pushed the xiangdong/dist_upstream_p1 branch from 1b3ead9 to 5b9e0a0 Compare July 30, 2025 03:59
@daisyden daisyden added the ciflow/xpu Run XPU CI tasks label Aug 8, 2025
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 8, 2025

To add the ciflow label ciflow/xpu please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

@pytorch-bot pytorch-bot bot removed the ciflow/xpu Run XPU CI tasks label Aug 8, 2025
@zxd1997066 zxd1997066 force-pushed the xiangdong/dist_upstream_p1 branch from c16839b to 47762fd Compare August 12, 2025 06:56
@guangyey guangyey moved this to Pre-Review Required in PyTorch Intel Aug 12, 2025
@guangyey guangyey added the ciflow/xpu Run XPU CI tasks label Aug 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guangyey guangyey guangyey left review comments

Assignees
No one assigned
Labels
ciflow/xpu Run XPU CI tasks module: xpu Intel XPU related issues oncall: distributed Add this issue/PR to distributed oncall triage queue open source topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
PyTorch Intel
Status: Pre-Review Required
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zxd1997066 @guangyey @pytorchbot @daisyden