Skip to content

Fixed #36485 -- Added linting tool for documentation files. #19549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

smithdc1
Copy link
Member

@smithdc1 smithdc1 commented Jun 9, 2025

Checking documentation in proposed patches for adherence to the 'style guide' takes time for the Fellows to review and update if required. The line length guide of 80 characters is a stand out here.

There's a tool called sphinx-lint, which err, lints sphix files. This tool's project repo seems to be fairly active and so seems well supported.

Unfortunately, it doesn't work out of the box for us as Django uses .txt extensions and the line length check needs some adjustments to be suitable for Django's docs.

This series of proposed patches does the following:

  1. Introduces the ability to lint the docs, along with some customisation to make it work with Django's docs.
  2. Applies the changes to the docs so they pass the linter
  3. Enforces the linter in CI.

It will need some polishing, and I'm happy to do that, if there's support for this change.

@github-actions github-actions bot added the no ticket Based on PR title, no linked Trac ticket label Jun 9, 2025
@nessita nessita self-requested a review June 10, 2025 14:55
Copy link
Contributor

@nessita nessita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some initial comments for first commit.

Copy link
Contributor

@nessita nessita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @smithdc1 to invest time in this work. I think we are in the right direction! 🌟

I was wondering if you consider making this an extension: we would still need some customization/wrangling, but we would avoid all the new Makefile rule/tox changes/etc etc.

I gave this a try and have this MVP (still needs the customizations). In an ideal scenario, this would replace commits 1 and 6:

commit 48075d9be469154fb26abde37b955a39c2785b4a (HEAD -> pr/19549)
Author: Natalia <124304+nessita@users.noreply.github.com>
Date:   Thu Jun 12 14:36:25 2025 -0300

    Alternative to explicit `lint` Makefile rule via Sphinx Extensions.

diff --git a/docs/_ext/djangolint.py b/docs/_ext/djangolint.py
new file mode 100644
index 0000000000..4009dc5524
--- /dev/null
+++ b/docs/_ext/djangolint.py
@@ -0,0 +1,28 @@
+from sphinx.util import logging
+from sphinxlint.checkers import all_checkers
+from sphinxlint.sphinxlint import check_text
+
+logger = logging.getLogger(__name__)
+
+
+def lint_file(app, docname, source):
+    path = app.env.doc2path(docname)
+    content = source[0]
+
+    checkers = list(all_checkers.values())
+    for checker in checkers:
+        if ".rst" in checker.suffixes:
+            checker.suffixes = (*checker.suffixes, ".txt")
+
+    errors = check_text(path, content, checkers)
+    for error in errors:
+        logger.error(error)
+
+
+def setup(app):
+    app.connect("source-read", lint_file)
+    return {
+        "version": "1.0",
+        "parallel_read_safe": True,
+        "parallel_write_safe": True,
+    }
diff --git a/docs/conf.py b/docs/conf.py
index be79b9133c..1836eed3df 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -44,6 +44,7 @@ needs_sphinx = "4.5.0"
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = [
     "djangodocs",
+    "djangolint",
     "sphinx.ext.extlinks",
     "sphinx.ext.intersphinx",
     "sphinx.ext.autosectionlabel",

@nessita
Copy link
Contributor

nessita commented Jun 12, 2025

For long lines that are links, there is an opened PR against the lint project: sphinx-contrib/sphinx-lint#132

@smithdc1
Copy link
Member Author

I was wondering if you consider making this an extension

I'm not sure. This would be a different approach taken with both spelling and black? Including this 'lint' step as part of the 'build' feels a bit strange to me, although I'm not sure I quite have the words to explain why. I'll have a go though. I think the benefits of a separate 'job' are:

  • Eases identification of where the error is. That is, you'd see the error on a github action rather than as a RTD failure. Reviewing logs on github is likely easier than needing to go over to RTD.
  • Once the error is fixed locally, you can just lint the docs. This would be much quicker than needing to do a full docs build.
  • If a lint error were to be merged (say?), then having a separate job for linting docs would not result in the docs builds failing at djangoproject.com.

@nessita
Copy link
Contributor

nessita commented Jun 20, 2025

I was wondering if you consider making this an extension

I'm not sure. This would be a different approach taken with both spelling and black? Including this 'lint' step as part of the 'build' feels a bit strange to me, although I'm not sure I quite have the words to explain why. I'll have a go though. I think the benefits of a separate 'job' are:

  • Eases identification of where the error is. That is, you'd see the error on a github action rather than as a RTD failure. Reviewing logs on github is likely easier than needing to go over to RTD.

  • Once the error is fixed locally, you can just lint the docs. This would be much quicker than needing to do a full docs build.

  • If a lint error were to be merged (say?), then having a separate job for linting docs would not result in the docs builds failing at djangoproject.com.

Thank you for this reasoning, these are all very valid points. My rationale was along these lines:

  • Running the linter as an extension and as part of the build could allow linting only the changed files, which IMHO is faster and more efficient than checking the entire docs tree every time, since for an existing _build folder, only the modified files are processed.
  • (Subjective) Contributors would get earlier and clearer feedback, they often don't realize that they need to run extra steps (make black spelling). If the check ran as part of the usual build process (or something closer to it), they’d get feedback earlier and with more context, rather than hitting CI failures they don't know how to interpret or fix.
  • CI failures are already a thing, since while it's true that a failing linter shouldn't block djangoproject.com docs builds, in practice other doc issues already cause build failures, like unresolved refs or broken class/method links. (A counterargument is that broken ref is a fatal error while a failed lint is not...)

Re: black and spelling, these two are somehow different already, spelling is a specific Sphinx builder while black a completely separated tool. Unclear where lint would fit, so in summary I'm open to your suggestion, but I did want us to explore the extension options before committing to a solution (to ensure we cover the basics and do a complete analysis).

@smithdc1
Copy link
Member Author

Thank you Natalia.

One other thing I thought about is having it available as a pre-commit hook. It seems that a separate repository would be required though and that feels a bit much. I think I'd rather try and discuss these changes with the sphinx-lint team and see if I can contribute the changes back upstream. This is a longer term goal though and shouldn't stop this work here.

Now that we've had a good discussion on the pro/cons of some different approaches I'm happy to help implement this change in the way that you recommend.

@nessita
Copy link
Contributor

nessita commented Jun 22, 2025

Now that we've had a good discussion on the pro/cons of some different approaches I'm happy to help implement this change in the way that you recommend.

My recommendation would be to push forward the approach that you think it's going to be less work for you, while providing useful feedback to the contributors. I sense that you may prefer the separated, non extension, lint rule and I'm happy if we go that path (as you said, we have covered on pro/cons and there is no clear "winner").

Perhaps a good next step for a small, self contained commit would be to add a new Makefile rule "check" that would do black, lint and spelling at once?

@sarahboyce sarahboyce changed the title Added linting tool for documentation files. Fixed #36485 -- Added linting tool for documentation files. Jul 31, 2025
@github-actions github-actions bot removed the no ticket Based on PR title, no linked Trac ticket label Jul 31, 2025
applications aren't ``washingtonpost.com`` or ``slashdot.org``; they're small-
to medium-sized sites with so-so traffic. But for medium- to high-traffic
applications aren't ``washingtonpost.com`` or ``slashdot.org``; they're
small-to medium-sized sites with so-so traffic. But for medium- to high-traffic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed a small typo in the commit msg: hypen -> hyphen, cheers! 👋

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch 🎯

Congrats on the role of Django Fellow! 🥳 🥳 🥳

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants