Skip to content

Allow to byteswap data when reading saved torch jit data #151447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Allow to byteswap data when reading saved torch jit data #151447

wants to merge 4 commits into from
+312 −12

Conversation

AlekseiNikiforovIBM
Copy link
Collaborator

@AlekseiNikiforovIBM AlekseiNikiforovIBM commented Apr 16, 2025 •
edited by pytorch-bot bot
Loading

It looks like some pickled data is endian-dependent. Byteswap such data when needed.

Add testcases.

Fixes #151428

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 16, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/151447

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 9103fc8 with merge base ab65581 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: jit release notes category label Apr 16, 2025
@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Apr 16, 2025
malfet
malfet requested changes Apr 16, 2025
View reviewed changes
Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not embed large files into .py, just store them as artifacts (it looks like you are adding nearly 1500 lines to test_save_load.py)

@AlekseiNikiforovIBM
Copy link
Collaborator Author

Sure, I'll update tests.

@AlekseiNikiforovIBM
Copy link
Collaborator Author

Tests are updated and build is fixed. Please take a look.

@albanD albanD added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Apr 17, 2025
@albanD albanD requested a review from mikaylagawarecki April 17, 2025 13:31
@mikaylagawarecki mikaylagawarecki requested review from davidberard98 and removed request for mikaylagawarecki May 1, 2025 18:39
@mikaylagawarecki
Copy link
Contributor

@davidberard98 tagging you for torch.jit (sorry, not sure whether you're still the right person here)

@AlekseiNikiforovIBM
Copy link
Collaborator Author

Could you please take a look at this PR again?

@AlekseiNikiforovIBM
Copy link
Collaborator Author

This change is still relevant.

@huydhn huydhn requested a review from malfet June 3, 2025 18:00
@AlekseiNikiforovIBM AlekseiNikiforovIBM force-pushed the torch_jit_load_byteorder branch from 9e564e7 to 9103fc8 Compare July 9, 2025 08:45
@AlekseiNikiforovIBM
Copy link
Collaborator Author

This PR is still relevant. Could you please review it?

@AlekseiNikiforovIBM AlekseiNikiforovIBM added the ciflow/s390 s390x-related CI jobs label Jul 9, 2025
@huydhn huydhn mentioned this pull request Aug 5, 2025
Closed
@albanD
Copy link
Collaborator

albanD commented Aug 5, 2025

Thank you very much for sending a PR.
To avoid you spending time preparing a PR for a feature we're not sure we want to add, our usual process is to discuss on the issue before doing the PR.
See https://github.com/pytorch/pytorch/wiki/The-Ultimate-Guide-to-PyTorch-Contributions for details!

In this particular case, I'm afraid no one is working on TorchScript so I'm not sure if anyone will fix this. Maybe @SherlockNoMad if this API is still used on your end?

@AlekseiNikiforovIBM
Copy link
Collaborator Author

The PR addresses an issue reported by a user. It is not adding any new functionality. Rather, it closes a gap for an otherwise well-supported platform. The code does not pose any risk to other platforms. The code is ready to be merged and includes test cases to ensure it continues to function properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidberard98 davidberard98 Awaiting requested review from davidberard98

@huydhn huydhn Awaiting requested review from huydhn

@seemethere seemethere Awaiting requested review from seemethere

@malfet malfet Awaiting requested review from malfet

Assignees
No one assigned
Labels
ciflow/s390 s390x-related CI jobs oncall: jit Add this issue/PR to JIT oncall triage queue open source release notes: jit release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

TorchScript Model Saved on x86 Returns NaNs When Loaded on s390x
6 participants
@AlekseiNikiforovIBM @mikaylagawarecki @albanD @malfet @facebook-github-bot @pytorchbot