Skip to content

Bug: Relative template_dir path causes TemplateNotFound error in monorepo configuration #845

New issue
New issue
Open
Open
Bug: Relative template_dir path causes TemplateNotFound error in monorepo configuration#845
Labels
bugSomething isn't working properlyconfirmedPrevent from becoming staleneeds-updateNeeds status update from maintainers
Milestone
Monorepo support
@deme3

Description

@deme3
deme3
opened on Feb 23, 2024

The problem

TemplateNotFound error thrown by Jinja2 when generating changelog in a subfolder. The directory structure is as follows:

- main_program
+-- templates
    +-- main_program
        +-- CHANGELOG.md.j2
+-- CHANGELOG.md
+-- pyproject.toml
+-- ...
- lib1
- lib2

To be clear, the templates are discovered by semantic-release, but when they're passed to Jinja, a relative path is used, and Jinja is not able to find them.

Expected behavior

I expect a generated CHANGELOG.md in the same level as pyproject.toml by using the templates provided in the configuration.

Environment

  • Python Version: v3.8.18
  • Pip Version: v23.0.1
  • Semantic Release Version: v9.1.0
pip freeze 
annotated-types==0.6.0
antlr4-python3-runtime==4.13.1
antlr4-tools==0.2
argcomplete==3.2.2
argparse-addons==0.12.0
asn1tools==0.166.0
bcrypt==4.0.1
Beaker==1.12.1
beautifulsoup4==4.12.2
bitstruct==8.17.0
build==1.0.3
cantools==39.0.0
capstone==5.0.0rc2
certifi==2024.2.2
cffi==1.15.1
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
colored-traceback==0.3.0
crccheck==1.3.0
cryptography==40.0.2
cssselect==1.2.0
decli==0.6.1
decorator==5.1.1
diskcache==5.6.1
docutils==0.20.1
dotty-dict==1.3.1
exceptiongroup==1.1.3
filebytes==0.10.2
FormEncode==2.1.0
future==0.18.3
gitdb==4.0.11
GitPython==3.1.42
idna==3.6
importlib-metadata==7.0.1
importlib-resources==6.1.1
iniconfig==2.0.0
install-jdk==1.1.0
intervaltree==3.1.0
jaraco.classes==3.3.1
jeepney==0.8.0
Jinja2==3.1.3
keyring==24.3.0
lark==1.1.8
-e git+<redacted>
lxml==4.9.3
Mako==1.2.4
markdown-it-py==3.0.0
MarkupSafe==2.1.2
mdurl==0.1.2
mikai==4.0.1
more-itertools==10.2.0
msgpack==1.0.5
-e git+<redacted>
nh3==0.2.15
nose==1.3.7
numpy==1.24.4
packaging==23.1
paramiko==3.1.0
Paste==3.7.1
PasteDeploy==3.1.0
PasteScript==3.3.0
pkginfo==1.9.6
pluggy==1.3.0
plumbum==1.8.1
prompt-toolkit==3.0.36
psutil==5.9.5
pwntools==4.9.0
pycparser==2.21
pydantic==2.6.1
pydantic_core==2.16.2
pyelftools==0.29
Pygments==2.16.1
Pylons==1.0.3
PyNaCl==1.5.0
pyparsing==3.1.0
pyproject_hooks==1.0.0
pyserial==3.5
PySide6==6.6.1
PySide6-Addons==6.6.1
PySide6-Essentials==6.6.1
PySocks==1.7.1
pytest==7.4.2
python-can==4.2.2
python-dateutil==2.8.2
python-gitlab==4.4.0
python-semantic-release==9.1.0
pytz==2023.3
PyYAML==6.0.1
questionary==2.0.1
readme-renderer==42.0
repoze.lru==0.7
requests==2.31.0
requests-toolbelt==1.0.0
rfc3986==2.0.0
rich==13.7.0
ROPGadget==7.3
ropper==1.13.8
Routes==2.5.1
rpyc==5.3.1
SecretStorage==3.3.3
shellingham==1.5.4
shiboken6==6.6.1
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
soupsieve==2.5
Tempita==0.5.2
termcolor==2.4.0
textparser==0.24.0
tomli==2.0.1
tomlkit==0.12.3
twine==5.0.0
typing_extensions==4.7.1
tzdata==2023.3
unicorn==2.0.1.post1
urllib3==2.2.1
waitress==2.1.2
wcwidth==0.2.9
WebError==0.13.1
WebHelpers==1.3
WebOb==1.8.7
WebTest==3.0.0
wrapt==1.15.0
zipp==3.17.0

Using build==1.0.3 and setuptools=56.0.0.

Configuration

[tool.semantic_release]
assets = []
commit_message = "chore: bumped version to {version}\n\nAutomatically generated by python-semantic-release"
commit_parser = "angular"
logging_use_named_masks = false
major_on_zero = true
tag_format = "main_program-v{version}"
version_variables = [
  "main_program/__init__.py:__version__"
]


[tool.semantic_release.changelog]
template_dir = "main_program/templates"
changelog_file = "CHANGELOG.md"
exclude_commit_patterns = []

Logs

GitHub Attachment to Logs

Additional context

There's something wrong here:

python-semantic-release/semantic_release/changelog/template.py

Lines 77 to 123 in 82bfcd3

def recursive_render(
template_dir: Path,
environment: Environment,
_root_dir: str | os.PathLike[str] = ".",
) -> list[str]:
rendered_paths: list[str] = []
for root, file in (
(Path(root), file)
for root, _, files in os.walk(template_dir)
for file in files
# we slice relpath.parts[1:] to allow the top-level
# template folder to have a dot prefix.
# e.g. to permit ".github/psr-templates" to contain the templates,
# rather than enforcing a top-level, non-hidden directory
if not any(
elem.startswith(".")
for elem in Path(root).relative_to(template_dir).parts[1:]
)
and not file.startswith(".")
):
output_path = (_root_dir / root.relative_to(template_dir)).resolve()
log.info("Rendering templates from %s to %s", root, output_path)
output_path.mkdir(parents=True, exist_ok=True)
if file.endswith(".j2"):
# We know the file ends with .j2 by the filter in the for-loop
output_filename = file[:-3]
# Strip off the template directory from the front of the root path -
# that's the output location relative to the repo root
src_file_path = str((root / file).relative_to(template_dir))
output_file_path = str((output_path / output_filename).resolve())
log.debug("rendering %s to %s", src_file_path, output_file_path)
stream = environment.get_template(src_file_path).stream()
with open(output_file_path, "wb+") as output_file:
stream.dump(output_file, encoding="utf-8")
rendered_paths.append(output_file_path)
else:
src_file = str((root / file).resolve())
target_file = str((output_path / file).resolve())
log.debug(
"source file %s is not a template, copying to %s", src_file, target_file
)
shutil.copyfile(src_file, target_file)
rendered_paths.append(target_file)
return rendered_paths

specifically, line 109

python-semantic-release/semantic_release/changelog/template.py

Lines 107 to 110 in 82bfcd3

log.debug("rendering %s to %s", src_file_path, output_file_path)
stream = environment.get_template(src_file_path).stream()

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working properlyconfirmedPrevent from becoming staleneeds-updateNeeds status update from maintainers

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions