Skip to content

Fix #155016 for Docathon - convert rst to markdown #155198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

j-silv
Copy link
Contributor

@j-silv j-silv commented Jun 5, 2025

Used rst2myst tool

One note is that "Created On" and "Last Updated On" banner doesn't show in the markdown files... I'm not sure if that's just an artifact of my local build though.

Fixes #155016

Docs comparison (check out the 'new' whenever docs build)

  1. cuda (old vs. new)
  2. cuda.tunable (old vs. new)
  3. leave cudnn_persistent_rnn.rst as is because it's reused in docstrings
  4. cudnn_rnn_determinism.rst as is because it's reused in docstrings.
  5. data (old vs. new)

cc @svekars @sekyondaMeta @AlannaBurke

Copy link

pytorch-bot bot commented Jun 5, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155198

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit f2ccb9c with merge base ee5c290 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions bot added docathon-h1-2025 medium Label for medium docathon tasks module: docs Related to our documentation, both in docs/ and docblocks triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jun 5, 2025
Copy link

pytorch-bot bot commented Jun 5, 2025

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: argument command: invalid choice: 'labels' (choose from 'merge', 'revert', 'rebase', 'label', 'drci', 'cherry-pick', 'close')

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci,cherry-pick,close} ...

Try @pytorchbot --help for more info.

Copy link

pytorch-bot bot commented Jun 5, 2025

Didn't find following labels among repository labels: module:,docs,topic:,not,user,facing,topic:,docs

@j-silv
Copy link
Contributor Author

j-silv commented Jun 5, 2025

@pytorchbot label docathon-h1-2025
@pytorchbot label module: docs
@pytorchbot label topic: not user facing
@pytorchbot label topic: docs

@j-silv
Copy link
Contributor Author

j-silv commented Jun 5, 2025

@pytorchbot label topic: not user facing

Copy link

pytorch-bot bot commented Jun 5, 2025

Didn't find following labels among repository labels: topic:,not,user,facing

@j-silv
Copy link
Contributor Author

j-silv commented Jun 5, 2025

@pytorchbot label "topic: not user facing"

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Jun 5, 2025
@j-silv
Copy link
Contributor Author

j-silv commented Jun 5, 2025

@pytorchbot label "topic: docs"

Copy link
Contributor

@svekars svekars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good overall - can you resolve the merge conflict?

@j-silv
Copy link
Contributor Author

j-silv commented Jun 5, 2025

  1. Ah I see. I was rebasing on viable/strict but it looks like cuda.rst was touched in the main branch in the meantime.

  2. Concerning your changes (i.e. adding python as the code block language), I think I could fork the rst2myst tool and add that functionality fairly easy. For now though, I'll just add the language keyword manually.

  3. Also, I think the fact that the "Created On" and "Last Updated On" banner don't show is actually a limitation of MySt, or there is a config option to be set.

  4. Finally, it seems that the rst2myst is really bad at handling anchor links.

For example, it converted this in data.rst:

* `single- and multi-process data loading <Single- and Multi-process Data Loading_>`_,

* `automatic memory pinning <Memory Pinning_>`_.

To this:

- [single- and multi-process data loading],
- [automatic memory pinning][memory pinning].

These are definitely not valid anchor links.

I saw that in torch.md, the author is using refs and in my case we would want:

- {ref}`map-style and iterable-style datasets <dataset-types>`,
- {ref}`customizing data loading order <data-loading-order-and-sampler>`,
- {ref}`automatic batching <loading-batched-and-non-batched-data>`,

and then wherever that header is, we would put:

(dataset-types)=
## Dataset Types

...etc

Just want to confirm that this is the style we should adopt. I tried to use explicit headers as specified per here, but those did not output correctly when running make html.

(heading-target)=
### Heading

[reference1](#heading-target)

@svekars
Copy link
Contributor

svekars commented Jun 6, 2025

thanks for providing these details:

  1. yes, please rebase to main
  2. 👍
  3. Created On | Last Updated On is a post-processing script we use - it should show. Let's get a preview for this PR to confirm
  4. Using {ref} sounds good.

@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from 83056f8 to dee4018 Compare June 6, 2025 20:09
@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from dee4018 to 1bae2c8 Compare June 6, 2025 22:49
Copy link
Collaborator

@albanD albanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python code change sounds good!

@svekars
Copy link
Contributor

svekars commented Jun 9, 2025

I think this won't work unfortunately since we probably can't use .md includes in docstrings themselves. Sorry for the confusion. We should undo the changes in the rnn.py files, and keep cudnn_rnn_determinism.rst, cudnn_persistent_rnn.rst as .rst files.

@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from 1bae2c8 to a997977 Compare June 9, 2025 22:48
@svekars
Copy link
Contributor

svekars commented Jun 10, 2025

It looks like it doesn't like the reference type of link in the docstring itself. You can change the ref to use a direct link instead:
multiprocessing context <https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods>_ in here and delete:

.. _multiprocessing context:
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

@svekars svekars self-requested a review June 10, 2025 17:29
@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from 8558661 to 0236637 Compare June 10, 2025 17:49
@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from 0236637 to 6e1d7c1 Compare June 10, 2025 17:59
@j-silv
Copy link
Contributor Author

j-silv commented Jun 10, 2025

Ahhh now the docstring is too wide:

  Advice (FLAKE8) B950
    line too long (174 > 120 characters)
    See https://github.com/PyCQA/flake8-bugbear#list-of-warnings

         175  |            worker subprocess with the worker id (an int in ``[0, num_workers - 1]``) as
         176  |            input, after seeding and before data loading. (default: ``None``)
         177  |        multiprocessing_context (str or multiprocessing.context.BaseContext, optional): If
    >>>  178  |            ``None``, the default `multiprocessing context <https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods>`_ of your operating system will

Any way around this?

@j-silv
Copy link
Contributor Author

j-silv commented Jun 11, 2025

One sec - data.md has no git log history. I'll break this into another commit to keep it

@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from b10e07f to 5663df7 Compare June 11, 2025 23:46
@svekars
Copy link
Contributor

svekars commented Jun 12, 2025

you can just undo your changes to data.rst on this branch (save them somewhere), then:

  1. From docs dir: git checkout origin/main -- source/optim.rst - delete .md I guess as well.
  2. git mv source/data.rst source/data.md
  3. Make a commit: git commit -m "Renamed source/data.rst to source/data.md".
  4. Can check git log: git log -- follow -- data.md - this should show you a bunch of commits.
  5. Push into this PR the above commit.
  6. Copy paste changes you've saved in step 1 into data.md. Commit/push into this PR.

@j-silv
Copy link
Contributor Author

j-silv commented Jun 12, 2025

Yep should be good now.

j-silv added 2 commits June 11, 2025 17:52
Used rst2myst tool: https://rst-to-myst.readthedocs.io/en/latest/
for the initial conversion and then hand-modified the broken links and
python syntax highlighting in the code blocks.

Squash svekars comments in the original PR.

Also this commit is broken into 2 so that we can keep the git log history of
data.md.
Second commit whereas the first was just a git mv to
maintain history of data.rst
@j-silv j-silv force-pushed the 155016-convert-to-markdown branch from 6356a57 to f2ccb9c Compare June 12, 2025 00:53
@svekars
Copy link
Contributor

svekars commented Jun 13, 2025

LGTM!
@pytorchbot merge -f "Dos change only, doc signal is green."

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docathon-h1-2025 medium Label for medium docathon tasks Merged module: docs Related to our documentation, both in docs/ and docblocks open source topic: docs topic category topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert to markdown: cuda.rst, cuda.tunable.rst, cudnn_persistent_rnn.rst, cudnn_rnn_determinism.rst, data.rst
5 participants