Skip to content

[C10d][Gloo] Enable complex datatype support in ProcessGroupGloo #156633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[C10d][Gloo] Enable complex datatype support in ProcessGroupGloo #156633

wants to merge 1 commit into from
+51 −22

Conversation

shunzhiwen
Copy link

@shunzhiwen shunzhiwen commented Jun 23, 2025 •
edited by pytorch-bot bot
Loading

  • Enable communication of tensors with Complex datatype in ProcessGroupGloo, similar to how ProcessGroupNCCL handles it.
  • Move a function, which checks if Complex datatype is supported by a reduce operation, from ProcessGroupNCCL.cpp into a new file to be shared with ProcessGroupGloo.

Fixes #156632

cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jun 23, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/156633

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 994d85d with merge base 414ad47 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@linux-foundation-easycla Linux Foundation: EasyCLA
Copy link

linux-foundation-easycla bot commented Jun 23, 2025 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: shunzhiwen / name: Shunzhi Wen (994d85d)

@pytorch-bot pytorch-bot bot added oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (c10d) release notes category labels Jun 23, 2025
@mikaylagawarecki mikaylagawarecki requested a review from d4l3k June 26, 2025 15:55
@mikaylagawarecki mikaylagawarecki added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jun 26, 2025
@shunzhiwen
Enable complex datatype support in ProcessGroupGloo
994d85d 
- Enable communication of tensors with Complex datatype in ProcessGroupGloo,
similar to how ProcessGroupNCCL handles it.
- Move a function, which checks if Complex datatype is supported by a reduce
operation, from ProcessGroupNCCL.cpp into a new file to be shared with
ProcessGroupGloo.

Signed-off-by: Shunzhi Wen <shunzhi.wen@arm.com>
@shunzhiwen
Copy link
Author

Renamed function to avoid conflicts with third party torch-xpu-ops. It defines a function with the same name under the same namespace in ProcessGroupXCCL.cpp.

d4l3k
d4l3k requested changes Aug 11, 2025
View reviewed changes
Copy link
Member

@d4l3k d4l3k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look very reasonable to me -- can we add some unit tests in test_c10d_gloo to check for complex type support?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@d4l3k d4l3k d4l3k requested changes

Assignees
No one assigned
Labels
oncall: distributed Add this issue/PR to distributed oncall triage queue open source release notes: distributed (c10d) release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support Complex Data Type in all_reduce and broadcast for Gloo Backend
4 participants
@shunzhiwen @d4l3k @pytorchbot @mikaylagawarecki