[Don't merge]port 2 distributed pipeline test files for Intel GPU #159140
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/159140
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 Cancelled Job, 2 Unrelated FailuresAs of commit 0b94210 with merge base aaa384b ( CANCELLED JOB - The following job was cancelled. Please retry:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "module: xpu" |
|
@pytorchbot label "triaged" |
| @@ -124,11 +131,18 @@ def test_stage_backward_weight(self, device): | |||
| ref_loss = loss_fn(ref_out, ref_target) | |||
| ref_loss.backward() | |||
|
|
|||
| rtol, atol = None, None | |||
| if self.device_type == "xpu": | |||
There was a problem hiding this comment.
having device specific logic in the tests is not ideal. Kind of curious, is this only hit for this test or should this logic be in the torch.testing.assert_close() util method?
Also if accuracy gap only happens for non-deterministic tests, can we just make the test deterministic?
There was a problem hiding this comment.
@wincent8 Let's try to set the deterministic option instead of tolerance.
There was a problem hiding this comment.
having device specific logic in the tests is not ideal. Kind of curious, is this only hit for this test or should this logic be in the
torch.testing.assert_close()util method?Also if accuracy gap only happens for non-deterministic tests, can we just make the test deterministic?
TBH, only these cases exhibit accuracy gaps in non-deterministic tests. For all other cases, XPU behaves as expected.
it's another pr to port distributed pipeline test for Intel GPU, while the other pr is #159033.
In this pr, we port two test files for Intel GPU
We could enable Intel GPU with following methods and try the best to keep the original code styles:
cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @gujinghui @EikanWang @fengyuan14 @guangyey