[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch #158644

FFFrog · 2025-07-18T12:22:06Z

Stack from ghstack (oldest at bottom):

To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components,
hoping to speed up the development of other accelerators.

This PR is the beginning of this note, and involve the part of registration of operators and we will gradually improve it and
keep in sync with OpenReg's code.

[ghstack-poisoned]

pytorch-bot · 2025-07-18T12:22:10Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158644

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 18 Pending

As of commit e4874ad with merge base ff0d56d ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 6b8c629 Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 7986da4 Pull-Request-resolved: #158644

albanD

Thanks for starting this!
I'm sure there is going to be a LOT of things to add there. But very exciting to see this first draft!

albanD · 2025-07-18T13:52:38Z

docs/source/conf.py

@@ -66,6 +66,7 @@
    "sphinx.ext.linkcode",
    "sphinxcontrib.mermaid",
    "sphinx_sitemap",
+    'sphinx_tabs.tabs',


@svekars is it ok to use this new extension?

albanD · 2025-07-18T13:52:59Z

docs/source/conf.py

@@ -82,6 +83,9 @@
 ]
 sitemap_url_scheme = "{link}"

+# todo options
+todo_include_todos = True


Isn't this going to show a lot of unrelated todos?

Hey @albanD

Sorry I forgot to change this Pr to a draft, I will remove all the todo later.

But I do need some advice from you. I will submit a simple RFC in the next few days. Please help me review whether the functional module scope of the document is insufficient/exceeding, whether the overall organization is reasonable, and whether the document can be placed in developer notes, etc.

There is another thing I need to ask for your opinion on, that is, there is a lot of work on this document and the supporting OpenReg code, so several colleagues will participate to speed up the overall speed. For the convenience of later development, can it be merged according to the module function, rather than merging after the entire doc document is completed?

albanD · 2025-07-18T13:59:46Z

docs/source/notes/accelerator.md

@@ -0,0 +1,274 @@
+```{eval-rst}


You need to add your new file to the index. Otherwise it's hard to get to from the website

Got it, thank you.

albanD · 2025-07-18T14:02:57Z

docs/source/notes/accelerator.md

+
+::::{tabs}
+
+:::{tab} C++


Note that the tab doesn't render well with the dark mode

PyTorch is very flexible and extensible, and we can extend it in many ways at the Python and C++ layers. To make the documentation clearer and easier to understand, I tried to use shpinx-tab to organize the documentation and divide some specific modules into two parts: Python and C++;

So, can we fix the abnormal display problem of sphinx-tab dark theme in pytorch_sphinx_theme2?

Please let me know, if there are other better ways to organize the documentation.

Yes tab sounds like a nice way to do this.
Feel free to send a PR to the theme repo to get this fixed !

albanD · 2025-07-18T14:13:16Z

docs/source/notes/accelerator.md

+The PrivateUse1-based third-party device integration mechanism has become the official path for integrating new devices into PyTorch. Ensuring the usability of this mechanism is crucial for enriching the hardware ecosystem of PyTorch.
+
+To assist third-party device developers in efficiently integrating new backends, this article introduces in detail the integration methods for typical PyTorch modules using a modular approach. It is accompanied by a streamlined code implementation from the official [torch_openreg][OpenReg URL] backend to help developers quickly get started while avoiding common pitfalls.
+
+This document is suitable for the following readers:
+
+* Developers who wish to integrate accelerator backends into PyTorch;
+* Developers interested in the principles of typical PyTorch modules;


I would suggest adding a first section explaining what we mean by backend quickly, the fact that this is all about eager, and the different pieces to consider (c10 integration, op implementations, python binding, testing, higher level modules (dist, autocast, etc)) and how Accelerator/PrivateUse1 makes your life easier.

If you're curious, you can take some of this more general context from https://docs.google.com/presentation/d/1rk3S0jqAPPBchZYkEle50p3fhMXAYBKj38jrFOJwlco/edit?slide=id.p34#slide=id.p34

Got it, I will add something on it and will cc you when it`s ready.

docs/source/notes/accelerator.md

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 9c3aee3 Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 9ea01fc Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 2e10e33 Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 275a5c1 Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 35f9292 Pull-Request-resolved: #158644

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 926cc15 Pull-Request-resolved: #158644

FFFrog · 2025-08-04T13:32:48Z

Hey, sorry to bother you @albanD.
The first version of the operator registration section of the documentation is ready. Please take a look. Thanks. (There's also some missing content, which we're currently working on.)

FFFrog · 2025-08-05T03:45:44Z

@albanD, can you help review and approve this PR into the trunk first, and then use this PR as a basis for subsequent PRs to submit documents and document-related code one by one, and split subsequent PRs into smaller PRs for easier review?

[ghstack-poisoned]

As the title stated. To facilitate the integration of the new backend, we plan to publish a new development note that details all the key components, hoping to speed up the development of other accelerators. This PR is the beginning of this note, which we will gradually improve and keep in sync with OpenReg's code. ghstack-source-id: 6f415c8 Pull-Request-resolved: #158644

[ghstack-poisoned]

Update

ae2d8f4

[ghstack-poisoned]

FFFrog requested review from jeffdaily and albanD as code owners July 18, 2025 12:22

pytorch-bot bot added the release notes: releng release notes category label Jul 18, 2025

This was referenced Jul 18, 2025

[OpenReg] Improve README.md and optimize some codes for OpenReg #158415

Closed

[OpenReg] add pyproject.toml for openreg #158440

Closed

[OpenReg] move fallback tests into test_openreg.py #158441

Closed

pytorchbot added the open source label Jul 18, 2025

Update

cb7c6e6

[ghstack-poisoned]

albanD reviewed Jul 18, 2025

View reviewed changes

janeyx99 reviewed Jul 18, 2025

View reviewed changes

docs/source/notes/accelerator.md Outdated Show resolved Hide resolved

Update

364df96

[ghstack-poisoned]

FFFrog marked this pull request as draft July 21, 2025 08:05

Update

d12bff1

[ghstack-poisoned]

FFFrog mentioned this pull request Jul 23, 2025

[Feat] Tracking for OpenReg Improvements #158917

Open

28 tasks

Update

103288c

[ghstack-poisoned]

Update

3cab36b

[ghstack-poisoned]

pytorch-bot bot added the topic: not user facing topic category label Aug 1, 2025

This was referenced Aug 1, 2025

Moved Autograd Fallback Interface to Header for Use by Out-of-tree Backends #159639

Open

[OpenReg] Refactor and Optimize the OpenReg for Preparation of Docs #159640

Open

FFFrog marked this pull request as ready for review August 1, 2025 09:14

FFFrog changed the title ~~[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch~~ [OpenReg] Add Develop Notes for Integrating New Backend into PyTorch(Operator Aspect) Aug 1, 2025

Update

a0fed8d

[ghstack-poisoned]

Update

e257353

[ghstack-poisoned]

Update

315cb15

[ghstack-poisoned]

FFFrog requested a review from ezyang August 7, 2025 01:46

Update

9362236

[ghstack-poisoned]

This was referenced Aug 7, 2025

[OpenReg] Add Event&Stream Support for OpenReg Backend #160099

Open

[OpenReg] Integrate Event&Stream from OpenReg Backend into PyTorch #160100

Open

[OpenReg] Improve the Event and Stream capabilities of DeviceGuardImplInterface #160101

Open

FFFrog added 2 commits August 8, 2025 18:32

Update

f7ecc8a

[ghstack-poisoned]

Update

e4874ad

[ghstack-poisoned]

FFFrog mentioned this pull request Aug 12, 2025

[OpenReg] Add OSX/Windows Support for OpenReg #160410

Closed

FFFrog changed the title ~~[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch(Operator Aspect)~~ [OpenReg] Add Develop Notes for Integrating New Backend into PyTorch Aug 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch #158644

[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch #158644

FFFrog commented Jul 18, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Jul 18, 2025 •

edited

Loading

Uh oh!

albanD left a comment

Uh oh!

albanD Jul 18, 2025

Uh oh!

albanD Jul 18, 2025

Uh oh!

FFFrog Jul 21, 2025

Uh oh!

albanD Jul 18, 2025

Uh oh!

FFFrog Jul 21, 2025

Uh oh!

albanD Jul 18, 2025

Uh oh!

FFFrog Jul 21, 2025

Uh oh!

albanD Jul 21, 2025

Uh oh!

albanD Jul 18, 2025

Uh oh!

FFFrog Jul 21, 2025

Uh oh!

Uh oh!

FFFrog commented Aug 4, 2025

Uh oh!

FFFrog commented Aug 5, 2025

Uh oh!

Uh oh!


		::::{tabs}

		:::{tab} C++

[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch #158644

Are you sure you want to change the base?

[OpenReg] Add Develop Notes for Integrating New Backend into PyTorch #158644

Conversation

FFFrog commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158644

⏳ No Failures, 18 Pending

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

FFFrog commented Aug 4, 2025

Uh oh!

FFFrog commented Aug 5, 2025

Uh oh!

Uh oh!

FFFrog commented Jul 18, 2025 •

edited

Loading

pytorch-bot bot commented Jul 18, 2025 •

edited

Loading