You should try out PR #1143, I haven't published it yet as it doesn't have automated testing but I have done significant testing with it on a monorepo example, https://github.com/codejedi365/psr-monorepo-poweralpha.

There is more improvements to make to the PR as I need to make squash commits available but this is a starting point. It isolates file paths and also allows for scope differentiated commits to create tags per submodule in the monorepo.

I'll take a look, wondering if a full parser is required, I just filtered commits based on scopes, which affected both release and changelog,

Is there any good reason for a new parser?

Thanks,

Asaf

Is there any good reason for a new parser?

Yes because I'm solving more than just your request for scope delineation. #614 request is more encompassing and my implementation includes scope as an additional filter.

Also I have other plans for the regular parsers to include scopes later but given the complexities of monorepos it's best to be separate. Monorepos tend to have multiple tag formats and I expect more issues in the future.

I see your point, I'm trying using to the new parser and I have some feedbacks:

Scope should be optional

If conventional commits defines the structure of <type>[optional scope]: <description>
Then in monorepo it would be <type>[<pkg>-optional scope]: <description>

So I would love to see the parser supports commit messages such as feat(srvc1): a minor bug fix where the scope would still be optional.

Another thing is that a root project is common in workspaces, so the main pyproject.toml if the monorepo is the project that uses the packages in the monorepo, IMO its commit could be none scoped, but it needs to filter out other packages commits so it must use the same parser.

I think the best way would be to use patterns rather scopes, that requires two props:
I'm not sure if it's possible to avoid the scope_optional because optional/non optional in regex is a hell.

scope_pattern: str = ""
    """
    A regex pattern that will be used to match the scope when parsing commit messages.

    If set, it will cause unscoped commits to be ignored. Use this in tandem with
    the path_filter option to filter commits by directory and scope.
    """

    scope_optional: bool = False
    """
    A boolean flag that determines whether the scope is optional or required.
    """

That way it's possible to define the root project such:

[tool.semantic_release.commit_parser_options]
scope_pattern = '(?!(?:srvc1|srvc2)(?:-|(?=\))))[\w-]+'
scope_optional = true

And in the packages:

[tool.semantic_release.commit_parser_options]
scope_pattern = 'srvsvc1(?:-[\w-]+)?'
scope_optional = false

Initial tests are working pretty well.

Changelog locations

The scripts are a bit tricky having pushd "documentation/templates" >/dev/null || exit, I assume this is temporary until this code is pushed?

I couldn't find a way to rename the changelog.md the CHANGELOG.md and make sure it's generated inside the packages instead of a separated folder.

I would love to get some guidelines how to do that.

Best,
Asaf.

Changelog locations

Firstly, it is important to note psr-monorepo-poweralpha built around the written description of another user's repository (I had to make it up to simulate). He was hosting all the documentation for the monorepo on a single site and likely through a documentation generator (like sphinx) which can be easiest to just point at a single directory. The root of the documentation for his project is documentation/docs so the sphinx command would be:

sphinx-autobuild documentation/docs documentation/docs/_build/html --open-browser
# this is notional as everything is in markdown and without a
# plugin/translator sphinx does not support markdown

Under the documentation/docs folder he would have a per service folder of documentation. If I remember correctly, he wanted some customized changelog templates or customized release notes which caused us to need a custom templates directory. If he didn't need it customized, then I think we could of just defined the setting changelog.default_templates.changelog_file to ../../../documentation/docs/srvc1/CHANGELOG.md in each of the submodule pyproject.toml files. The customized changelogs were not actually different between submodules so we consolidated all the template files into a base directory (documentation/templates/.base_changelog_template). Then during the CI process, prior to releasing a specific service, the base template directory would be recursively copied to the same directory but instead with the proper service name and the full path that would be the destination location from the root of the repository. The base changelog template directory is a hidden folder because we had to set the documentation/templates directory as the template_dir for each of the submodules in order for the release notes template to work properly.

The scripts are a bit tricky having pushd "documentation/templates" >/dev/null || exit, I assume this is temporary until this code is pushed?

No, this is valid and proper defensive bash programming technique. I recommend looking into ShellCheck as it is a bash linting tool and it has a rule to recommend that all change directory commands should abort the script if a change directory fails. pushd is a command to change directory but maintain the previous location in a stack to enable return to the directory with popd. It is common to put the output of the commands to null because it prints the entire new stack by default which clutters script output. I used pushd to simplify the following command parameters with a controlled current directory change. You can also add if statements for each of the commands but that is more clutter than an else (||) exit.

I couldn't find a way to rename the changelog.md the CHANGELOG.md and make sure it's generated inside the packages instead of a separated folder.

If you want it generated inside the packages then just remove the tool.semantic_release.changelog.template_dir setting of both services. This will trigger the internal default template to be used and since it is intended to run semantic-release with the current working directory as the submodule directory this will write the changelog as normal. If you want a customized changelog, that is a bit more complicated.

Uh, I thought the current changelog would not be sufficient in case of a monorepo and that's your minimal repo for the parser demonstration, fair enough, for a minimal example I will clean this up so others can have a simple reference.

Uh, I thought the current changelog would not be sufficient in case of a monorepo and that's your minimal repo for the parser demonstration, fair enough, for a minimal example I will clean this up so others can have a simple reference.

You are expecting things that are not produced or published yet. This is why PSR does not officially support monorepos. I gave you an example of what I have at this point. Their monorepo was complex and kept getting more complex as I provided an example and then had to re-design it. If you want to provide a recommendation of a simpler monorepo, then I would appreciate it. The nuance of your repo design is the inclusion of a top level pyproject.toml. I'm not sure how you made that work as python is very picky about project structure.

Ye I get it :), thanks for the quick response, I appreciate it! I'm working on a blog post about how to release a uv workspace with PSR and I'll link it here for future reference.

Thanks.

Thanks @asaf for taking the time to provide feedback.

Scope should be optional

If conventional commits defines the structure of <type>[optional scope]: <description> Then in monorepo it would be <type>[<pkg>-optional scope]: <description>

So I would love to see the parser supports commit messages such as feat(srvc1): a minor bug fix where the scope would still be optional.

I disagree with your concept here. Monorepos are not defined in the conventional-commits spec so stating which service its related to is still a provided scope. Secondly, the parser I recommended filters by file paths as the primary mechanism and the scope is secondary. You can still provide no scope and as long as the files are in the correct module then it will be selected for the changelog. The real magic is for changes to files outside of the submodule that are still relevant to the project then you can specify it via the scope prefix. In this case, it is my opinion that you should also specify a deeper scope of what external file/process is affecting the version bump of the submodule.

Another thing is that a root project is common in workspaces, so the main pyproject.toml if the monorepo is the project that uses the packages in the monorepo, IMO its commit could be none scoped, but it needs to filter out other packages commits so it must use the same parser.

Although conceptually this makes sense, I haven't seen python actually able to support a top level project with submodules. I have considered a top level configuration rather than the expectation of multiple configurations within each submodule but that is a ways off. If you had a top level semantic-release configuration with the same parser, it would detect file changes first and you would just need to add a path negation (!) to the path_filters for the folder that contains the submodules. It should work from there. It would probably have a hiccup though, if you had files at the top level that had a scope for the submodules that weren't ignored by the path_filter.

Will have to reconsider top level releasing once I have an example of it.

Initial tests are working pretty well.

Glad to hear that.

Ye I get it :), thanks for the quick response, I appreciate it! I'm working on a blog post about how to release a uv workspace with PSR and I'll link it here for future reference.

sounds good.

I disagree with your concept here. Monorepos are not defined in the conventional-commits spec so stating which service its related to is still a provided scope. Secondly, the parser I recommended filters by file paths as the primary mechanism and the scope is secondary. You can still provide no scope and as long as the files are in the correct module then it will be selected for the changelog. The real magic is for changes to files outside of the submodule that are still relevant to the project then you can specify it via the scope prefix. In this case, it is my opinion that you should also specify a deeper scope of what external file/process is affecting the version bump of the submodule.

I had to add IMO regarding the package scopes :) I think it's reasonable to have simple typed commits for a scoped package. it clarifies which commit belongs to which package.

I think my primary misunderstanding here is that I thought filtering is all about scopes.

If filtering is primarily done by files (that also justifies a full parser rather the PR I've sent for a filter by scope) then removing the prefix will do for me.

Although conceptually this makes sense, I haven't seen python actually able to support a top level project with submodules. I have considered a top level configuration rather than the expectation of multiple configurations within each submodule but that is a ways off. If you had a top level semantic-release configuration with the same parser, it would detect file changes first and you would just need to add a path negation (!) to the path_filters for the folder that contains the submodules. It should work from there. It would probably have a hiccup though, if you had files at the top level that had a scope for the submodules that weren't ignored by the path_filter.

I think uv treat root package as just another package in the workspace,
I'll give path_filters a try and see how it goes, the other way to bypass all this hassle is to put the "root" package under packages.

@asaf, thanks for the details. I hope the path filter parser from PR #1143, requested in #614, is fitting your use case. I'm sorry that I don't have documentation prepared already nor a simpler monorepo for demonstration.

This issue has not received a response in 14 days. If no response is received in 7 days, it will be closed. We look forward to hearing from you.

This issue was closed because no response was received.