Skip to content

Fix error on importing unavailable torch.distributed #40038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix error on importing unavailable torch.distributed #40038

wants to merge 1 commit into from

Conversation

m-gallus
Copy link

@m-gallus m-gallus commented Aug 8, 2025

What does this PR do?

Currently PyTorch on Windows builds don't support distributed module and when users attempt to use transformers or a lib dependent on it, it fails with the following error:

File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\tensor\__init__.py", line 4, in <module>
    import torch.distributed.tensor._ops  # force import all built-in dtensor ops
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\tensor\_ops\__init__.py", line 2, in <module>
    from ._conv_ops import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\tensor\_ops\_conv_ops.py", line 5, in <module>
    from torch.distributed.tensor._dtensor_spec import DTensorSpec, TensorMeta
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\tensor\_dtensor_spec.py", line 6, in <module>
    from torch.distributed.tensor.placement_types import (
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\tensor\placement_types.py", line 8, in <module>
    import torch.distributed._functional_collectives as funcol
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\_functional_collectives.py", line 9, in <module>
    import torch.distributed.distributed_c10d as c10d
  File "C:\Users\Micha\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\distributed\distributed_c10d.py", line 23, in <module>
    from torch._C._distributed_c10d import (
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package

This is caused by model_debugging_utils.py having an unguarded import torch.distributed.tensor. This PR ensures that the distributed module is available before including its tensor module.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@ArthurZucker

@m-gallus
Fix error on importing unavailable torch.distributed
bc65790 
Currently model_debugging_utils.py would have an unguarded `import torch.distributed.tensor`. This PR ensures that the distributed module is available before including its tensor module.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@m-gallus