Skip to content

Infra: generate machine-readable PEP index #2475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 1, 2022
Merged

Infra: generate machine-readable PEP index #2475

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
36b2a5f
Infra: generate machine-readable PEP index
hugovk Mar 27, 2022
c24b85b
Don't create a temp list
hugovk Mar 29, 2022
30d0d6c
Infra: Include more headers in peps.json
hugovk Mar 29, 2022
089826a
Infra: Move from /peps.rss to /api/peps.json
hugovk Mar 29, 2022
5f88ed5
Infra: Build JSON from dict, with PEP number as key, instead of list
hugovk Mar 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions pep_sphinx_extensions/pep_zero_generator/parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,20 @@ def __init__(self, filename: Path, authors_overrides: dict):
# Parse PEP authors
self.authors: list[Author] = _parse_authors(self, metadata["Author"], authors_overrides)

# Other headers
self.created = metadata["Created"]
self.discussions_to = metadata["Discussions-To"]
self.python_version = metadata["Python-Version"]
self.replaces = metadata["Replaces"]
self.requires = metadata["Requires"]
self.resolution = metadata["Resolution"]
self.superseded_by = metadata["Superseded-By"]
if metadata["Post-History"]:
# Squash duplicate whitespace
self.post_history = " ".join(metadata["Post-History"].split())
else:
self.post_history = None

def __repr__(self) -> str:
return f"<PEP {self.number:0>4} - {self.title}>"

Expand Down
33 changes: 30 additions & 3 deletions pep_sphinx_extensions/pep_zero_generator/pep_index_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from __future__ import annotations

import csv
import json
from pathlib import Path
import re
from typing import TYPE_CHECKING
Expand All @@ -30,9 +31,29 @@
from sphinx.environment import BuildEnvironment


def create_pep_zero(_: Sphinx, env: BuildEnvironment, docnames: list[str]) -> None:
# Sphinx app object is unneeded by this function

def create_pep_json(peps: list[parser.PEP]) -> str:
pep_dict = {
pep.number: {
"title": pep.title,
"authors": ", ".join(pep.authors.nick for pep.authors in pep.authors),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to make this an array of nicks, so users don't need to split the string again?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, all of the fields are strings (or null), and match the literal value of the headers (minus cleaned-up whitespace). if we're going to do this, it would be inconsistent if we didn't process the other headers as well into more structured formats, which would be a good idea but better left to a future PR. With #2484 , tools can rely on each of the headers to match a certain format, so they are easier and more consistent to work with as strings and can be split with just .split(","), without having to worry much about edge cases.

As mentioned above,

In the future, if there's any interest, we could actually parse the fields into more structured formats, especially once my PRs land that fully validate their format for much easier and trouble-free internal and external use.

"discussions_to": pep.discussions_to,
"status": pep.status,
"type": pep.pep_type,
"created": pep.created,
"python_version": pep.python_version,
"post_history": pep.post_history,
"resolution": pep.resolution,
"requires": pep.requires,
"replaces": pep.replaces,
"superseded_by": pep.superseded_by,
"url": f"https://peps.python.org/pep-{pep.number:0>4}/",
}
for pep in sorted(peps)
}
return json.dumps(pep_dict, indent=1)


def create_pep_zero(app: Sphinx, env: BuildEnvironment, docnames: list[str]) -> None:
# Read from root directory
path = Path(".")

Expand Down Expand Up @@ -63,3 +84,9 @@ def create_pep_zero(_: Sphinx, env: BuildEnvironment, docnames: list[str]) -> No
docnames.insert(1, pep_zero_filename)
# Add to files for writer
env.found_docs.add(pep_zero_filename)

# Create peps.json
pep0_json = create_pep_json(peps)
out_dir = Path(app.outdir) / "api"
out_dir.mkdir(exist_ok=True)
Path(out_dir, "peps.json").write_text(pep0_json, encoding="utf-8")