Skip to content

Make every formatter optional #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 8, 2022

Conversation

Pierre-Sassoulas
Copy link
Collaborator

Work in progress, this still need more robust tests. This would permit to automatically generate the arguments and to activate or deactivate every parser while still having a default.

@Pierre-Sassoulas Pierre-Sassoulas added the enhancement New feature or request label Jan 4, 2022
@coveralls
Copy link

coveralls commented Jan 4, 2022

Pull Request Test Coverage Report for Build 1671474603

  • 23 of 24 (95.83%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.5%) to 99.537%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pydocstringformatter/utils/argument_parsing.py 19 20 95.0%
Totals Coverage Status
Change from base Build 1669117841: -0.5%
Covered Lines: 215
Relevant Lines: 216

💛 - Coveralls

@Pierre-Sassoulas Pierre-Sassoulas changed the base branch from main to refactor-formatter-module January 4, 2022 08:30
Copy link
Owner

@DanielNoord DanielNoord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also an empty file pydocstringformatter/formatting/argparse_registerer.py.

I do like this idea though! What about adding an option_group in argparse? That way they will be separated in the help message and we have some more control over what to display and what not.

@Pierre-Sassoulas Pierre-Sassoulas force-pushed the refactor-formatter-module branch from a3ee101 to 64774ee Compare January 4, 2022 13:39
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 74da6f7 to 59be4e7 Compare January 4, 2022 13:59
Base automatically changed from refactor-formatter-module to main January 4, 2022 15:38
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 59be4e7 to 513bdc1 Compare January 4, 2022 15:40
@Pierre-Sassoulas Pierre-Sassoulas added this to the 0.3.0 milestone Jan 4, 2022
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 513bdc1 to 92ef78d Compare January 4, 2022 18:46
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 717fe63 to ce641b1 Compare January 4, 2022 20:56
@Pierre-Sassoulas Pierre-Sassoulas changed the base branch from main to add-name-to-formatters January 4, 2022 20:56
Base automatically changed from add-name-to-formatters to main January 4, 2022 21:08
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from ce641b1 to 691ab8a Compare January 4, 2022 21:09
f"--{name}",
action="store_true",
dest=name,
default=not formatter.optional,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have thought this a little more and I think it would be better if we removed the default here and did another _register_formatter_defaults function. The documentation is pretty clear about what store_true is "supposed" to be used for:
'store_true' and 'store_false' - These are special cases of 'store_const' used for storing the values True and False respectively. In addition, they create default values of False and True respectively.

Just to be sure for future changes I think it makes more sense to honour the default values created by store_true and then parse the optional attribute. That also makes more sense for any outsider looking at this code for the first time.

What do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I don't understand exactly what you propose. Could you adapt what we currently have ?

parser.add_argument(
    f"--{name}",
    action="store_true"
    default=not optional,
)
parser.add_argument(
    f"--{name}",
    action="store_false",
)    

It seems that if we remove the default value then we need additional parsing later on ? Can we guess if the value is user defined or simply the default ?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need some sort of parsing later on anyway to make sure the default value actually gets into the Namespace object. I think it would be better to separate that parsing (and thus the parsing of the Formatter.optional attribute into its own function) than setting the default value here to a value that can be counter-intuitive to what the documentation says.

As for parsing and whether it is user defined I think this dependent on the order in which we parse.
I would suggest: defaults > config files > command line.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But right now we don't have to parse anything as the default override what store_const is doing (logically). If we create config file and need another layer we could use confuse to do something like this:

config: confuse.Configuration
config_default = config[name].get(confuse.Optional("bool"))
default = not optional if config_default is None else config_default 
parser.add_argument(
    f"--{name}",
    action="store_true"
    default=default 
)
parser.add_argument(
    f"--{name}",
    action="store_false",
) 

Also this is a part of the code that can almost never change as it's decoupled and generic, we don't have to touch it for every new formatter.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have never worked with confuse so I don't know specifically what it does here.

I understand how the current format makes the code more efficient and achieve the same with less lines, but I think in the long run it only adds maintainability costs to the project: every peculiar thing a project does that is not "as expected" increases the cost for new contributors to understand what is happening. To me it makes much more sense to add two options using store_false and store_true and then parse a default value to store the correct value in dest. To me that feels more like the natural flow of argparse. However, if you're insistent on this we can keep your suggestion 😄

Copy link
Collaborator Author

@Pierre-Sassoulas Pierre-Sassoulas Jan 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confuse is pretty nice let me show you with a confuse example :)

So confuse handles the overriding of configuration (pretty intuitively, as it's what the lib does). There is a twitch for us : When do we recover our own default value from the formatter ? So we do not define a default value in our own yaml config ,so if confuse has something it must comes from the user. After that the following line: default = not optional if config_default is None else config_default actually does "if the configuration from confuse is not specified than the default comes from our formatter".

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm confuse looks nice, but is it really necessary for us? We don't really need to look for config files in usr directories (right now) and I think adding too many dependencies to such a simple auto-formatter might make it less compatible with other apps.

In the end I really like the approach of --formatter and --no-formatter (I didn't think of that myself), but I'd just like 1) not to overwrite the default value for a store_true option and 2) keep option adding and option parsing separate so we can easily iterate on them and know what is responsible for what.

@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 8ff1a6c to bfdcd14 Compare January 5, 2022 10:18
@Pierre-Sassoulas Pierre-Sassoulas changed the base branch from main to add-prettier-to-pre-commit January 5, 2022 10:19
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the add-prettier-to-pre-commit branch 2 times, most recently from fb28775 to 342f3e7 Compare January 5, 2022 10:20
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from dde6cca to 6fde3df Compare January 8, 2022 09:01
Pierre-Sassoulas pushed a commit that referenced this pull request Jan 8, 2022
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 6fde3df to c1e7de3 Compare January 8, 2022 09:30
@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from c55b393 to b74f5fd Compare January 8, 2022 09:53
@Pierre-Sassoulas
Copy link
Collaborator Author

I added the last commit to be able to cover the case where an optional formatter was used (we don't have any in FORMATTERS yet). Previously it was handled in argparse with the default value so did not need to be tested. I guess it would permit to expose the Run function to the outside so user can define their own formatters if we want. I can't separate it from this MR easily but we could do it later if we find another way to cover the creation of argument for optional formatter.

@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 8c669d2 to a2b9805 Compare January 8, 2022 12:08
Copy link
Owner

@DanielNoord DanielNoord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there!

Personally I would be okay with passing this with the decreased coverage and not adding the non-standard formatter option just yet. I'd like to make #25 optional to begin with, as I think it might be quite "opinionated" and would require some refactoring for most projects. (For example, I wonder how strictly we currently adhere to this in pylint).
When I implement #25 I could add tests for optional = True cases and increase the coverage to 100% again.

Let me know what you think!

@DanielNoord
Copy link
Owner

Btw, after this we should probably create a follow-up issue to create something like enable, disable in pyproject.toml.

@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from acb6fa5 to 11e137d Compare January 8, 2022 13:13
@Pierre-Sassoulas
Copy link
Collaborator Author

I would be okay with passing this with the decreased coverage

That would be a way to not have to mix the change in Run's API with this one. I think it's a configuration in coverage to change ?

@DanielNoord
Copy link
Owner

I would be okay with passing this with the decreased coverage

That would be a way to not have to mix the change in Run's API with this one. I think it's a configuration in coverage to change ?

Yeah, I set it to disallow any negative change in coverage. But since it isn't a required action I think you can merge while the coverage actions fails right? I would be fine with just letting coverage drop on this one. Alternatively you could add a TODO linking to #25 and set a pragma: no cover.

Copy link
Owner

@DanielNoord DanielNoord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final changes!

Thanks @Pierre-Sassoulas. I was quite insistent on some changes in this PR, but I'm really happy with the final system you/we came up with. This feels like it really suits our needs but is also easily expandable. Thanks for your help!

msg = "Nothing was modified, but all formatters are activated."
assert "Nothing to do!" not in out
expected = ["---", "@@", "+++"]
assert all(e in out for e in expected), msg
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite a neat way to test changes occurred! Might use this myself for future tests as well!

Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com>
@DanielNoord
Copy link
Owner

Oops, I made a typo in my suggestion. Sorry about that!

@Pierre-Sassoulas Pierre-Sassoulas force-pushed the make-every-formatter-optional branch from 495629c to 38a4967 Compare January 8, 2022 14:18
namespace: argparse.Namespace,
formatters: List[Formatter],
) -> None:
"""Parse the state of a the list of formatters based on their 'optional' attribute."""
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Parse the state of a the list of formatters based on their 'optional' attribute."""
"""Parse the state of a list of formatters based on their 'optional' attribute."""

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to apply that locally anyway because the docstring was too long for flake8 :)

@Pierre-Sassoulas
Copy link
Collaborator Author

Good design need some heated discussion me think 😄

@Pierre-Sassoulas Pierre-Sassoulas merged commit 0ded93a into main Jan 8, 2022
@Pierre-Sassoulas Pierre-Sassoulas deleted the make-every-formatter-optional branch January 8, 2022 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants