Skip to content

gh-130645: Add colour to argparse help #132323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Apr 9, 2025

Add color parameter to argparse.ArgumentParser, which by default is False. When set to True, the help text is in colour if allowed (for example, we're not piping, NO_COLOR=1 is not set).

Example output

Using this script:

import argparse

parser = argparse.ArgumentParser(
    description="calculate X to the power of Y",
    formatter_class=argparse.ArgumentDefaultsHelpFormatter,
    prefix_chars="-+",
    prog="PROG",
)
parser.color = True
group = parser.add_mutually_exclusive_group()
group.add_argument("-v", "--verbose", action="store_true", help="more spam")
group.add_argument("-q", "--quiet", action="store_true", help="less spam")
parser.add_argument("x", type=int, help="the base")
parser.add_argument("y", type=int, help="the exponent", deprecated=True)
parser.add_argument(
    "this_indeed_is_a_very_long_action_name", type=int, help="the exponent"
)
parser.add_argument("-o", "--optional1", action="store_true", deprecated=True)
parser.add_argument("--optional2", help="pick one")
parser.add_argument("--optional3", choices=("X", "Y", "Z"))
parser.add_argument("--optional4", choices=("X", "Y", "Z"), help="pick one")
parser.add_argument("--optional5", choices=("X", "Y", "Z"), help="pick one")
parser.add_argument("--optional6", choices=("X", "Y", "Z"), help="pick one")
parser.add_argument(
    "-p", "--optional7", choices=("Aaaaa", "Bbbbb", "Ccccc", "Ddddd"), help="pick one"
)

parser.add_argument("+f")
parser.add_argument("++bar")
parser.add_argument("-+baz")
parser.add_argument("-c", "--count")

subparsers = parser.add_subparsers(
    title="subcommands", description="valid subcommands", help="additional help"
)
subparsers.add_parser("sub1", deprecated=True, help="sub1 help")
sub2 = subparsers.add_parser("sub2", deprecated=True, help="sub2 help")
sub2.add_argument("--baz", choices=("X", "Y", "Z"), help="baz help")

args = parser.parse_args()
image

Performance

With the same script and PGO+LTO on macOS, running it once (time ./python.exe argparse-cli.py --help) and 100 times (time ./100.sh):

# 100.sh
for i in $(seq 1 100); do
    ./python.exe argparse-cli.py -h
done

main (3feac7a):

./python.exe argparse-cli.py --help  0.03s user 0.01s system 88% cpu 0.046 total
./100.sh  1.51s user 0.39s system 93% cpu 2.041 total

PR and colour not enabled:

./python.exe argparse-cli.py --help  0.02s user 0.02s system 58% cpu 0.072 total
./100.sh  1.52s user 0.40s system 92% cpu 2.070 total

PR and colour enabled:

./python.exe argparse-cli.py --help  0.04s user 0.02s system 70% cpu 0.080 total
./100.sh  1.57s user 0.40s system 93% cpu 2.106 total

Something like 0.65 ms difference per run between before and after, averaging from 100.sh.


📚 Documentation preview 📚: https://cpython-previews--132323.org.readthedocs.build/

@hugovk hugovk added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 9, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @hugovk for commit 3d174b7 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F132323%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 9, 2025
@hugovk hugovk added type-feature A feature request or enhancement stdlib Python modules in the Lib dir labels Apr 9, 2025
@hugovk hugovk marked this pull request as ready for review April 9, 2025 18:12
@python-cla-bot
Copy link

python-cla-bot bot commented Apr 18, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

indent_increment=2,
max_help_position=24,
width=None,
prefix_chars='-',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming you just wanted to add prefix_chars to the constructor for consistency reasons and I'm not missing something else, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do need to pass the prefix chars from the ArgumentParser to the HelpFormatter, so we can distinguish short options from long options.

For example, with this:

import argparse

parser = argparse.ArgumentParser(prefix_chars="-+", color=True)
parser.add_argument("+a")
parser.add_argument("-b")
parser.add_argument("++bleep")
parser.add_argument("-+bloop")
parser.add_argument("-c", "--count")
parser.add_argument("-d", "-+donut")

args = parser.parse_args()

Commenting out line 2735, we don't recognise +a as a short option, don't recognise ++bleep as a long option, and misrecognise -+bloop and -+donut as short options:

image

With this PR, we get:

image


if color and can_colorize():
self._ansi = ANSIColors()
self._decolor = decolor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having to do this, couldn't we just set color enabled in the constructor and then also define something like:

def color_me(text, some_color):
  if color:
    return f"{some_color}{text}"
  return text

...Then wrap all of the places we want to colorize? I think that way we could remove the ANSI color stripping and make this a bit easier to follow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One place we're using this _decolor is in _format_usage for calculating the offsets.

We've already processed all the bits and constructed a string, and added colour relevant for each bit. Here's what the full string looks like:

image

We then want to post-process it check:

  1. will it even fit on one line
  2. if not, calculate the offset for each line, based on the offset for the "usage" prefix and prog name:
image

If we had not already added colour, we would have to somehow figure out where to insert the colour, which I think would be more complicated.

But because we already have colour, we need to make sure the ANSI colour characters aren't included in the width calculations. It seems simpler to use the len() of a colour-stripped string, than the len() of the one with colour.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other place we#re using _decolor is in calclating the padding of actions in _format_action.

So given desired output like:

image

We need to calculate the red lines here, the gap between the action_header (for example, x) and the help (for example, the base):

image

There are three cases:

  1. No help: easy, we don't need to calculate a gap. Colour doesn't matter.
  2. Long action name: fairly easy, action goes on one line, help on a new line. Help has no colour, we don't need to worry about colour codes when calculating the offset.
  3. Short action name: bit harder: action and help fit on one line.

We're using this formatter:

            tup = self._current_indent, '', action_width, action_header
            action_header = '%*s%-*s  ' % tup

%*s means take a min width from the first value -- self._current_indent -- and use it with the next value -- '' -- so just apply the indent. This is okay.

%-*s is similar but means left-align. action_width is the space we have to left align and action_header is something like x. But if we have ANSI codes, we'll count some of these in the width and the gap will be too short.

https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting

So we need to calculate this without colour too. Again, some of these strings can have multiple colours applied, so it seems easier to remove the codes when calculating widths, rather than somehow needing to reconstruct the string or re-insert colour.

However, I've noticed I can make this particular change more focused, only for this third case, so I've committed that (c0a0688).

@bedevere-app
Copy link

bedevere-app bot commented Apr 21, 2025

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Copy link
Member

@savannahostrowski savannahostrowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment about the approach and whether we can simplify a bit.

@hugovk
Copy link
Member Author

hugovk commented Apr 30, 2025

ArgumentParser() has a lot of parameters, color is its 16th.

Many argparse CLIs I see already use keywords for most parameters, but I think we should make these new ones keyword-only, especially for new bools, as a long list of (..., True, False, True, ...) is best avoided.

suggest_on_error was added in 3.14, I suggest we also make that keyword-only (in another PR).

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting changes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants