argparse: subparsers, argument abbreviations and ambiguous option #58573

jakub · 2012-03-18T20:28:00Z

BPO	14365
Nosy	@merwok
Files	argparse_subparsers_ambiguous_bug.py: Bug reproduction argparse_dirty_hack.py subparser_optionals.diff subparser_patch.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-03-18.20:27:59.700>
labels = ['type-bug', 'library']
title = 'argparse: subparsers, argument abbreviations and ambiguous option'
updated_at = <Date 2014-08-07.22:47:31.748>
user = 'https://bugs.python.org/jakub'

bugs.python.org fields:

activity = <Date 2014-08-07.22:47:31.748>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2012-03-18.20:27:59.700>
creator = 'jakub'
dependencies = []
files = ['24928', '24945', '26466', '31901']
hgrepos = []
issue_num = 14365
keywords = ['patch']
message_count = 14.0
messages = ['156272', '156279', '156280', '156290', '156302', '156304', '156306', '156352', '166059', '198456', '198507', '198508', '198563', '225044']
nosy_count = 5.0
nosy_names = ['bethard', 'eric.araujo', 'tshepang', 'paul.j3', 'jakub']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14365'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

Linked PRs

jakub · 2012-03-18T20:27:59Z

Assuming following:

optional argument, say "--foo", in a subparser;
at least two different optional arguments in the main parser prefixed with "--foo", say "--foo1" and "--foo2";

parsing fails with "error: ambiguous option: --foo could match --foo1, --foo2".

It seems like argument abbreviation mechanism described at http://docs.python.org/library/argparse.html#argument-abbreviations is not working properly when in use with subparsers...

tshepang · 2012-03-18T22:21:07Z

More or less a duplicate of 12713?

tshepang · 2012-03-18T22:21:28Z

Sorry, I meant bpo-12713.

bethard · 2012-03-18T23:16:58Z

Yep. Closing as duplicate.

jakub · 2012-03-19T01:27:26Z

I don't understand how both bugs are related. Surely, patch provided for bpo-12713 does not fix the issue I described.

bethard · 2012-03-19T01:39:04Z

My mistake. I see that the error you're getting is a bad interaction between the option in the main parser and an ambiguous option in the subparser.

bethard · 2012-03-19T01:51:54Z

The problem is basically that _parse_known_args calls _parse_optional to determine whether something is an optional or a positional. But _parse_optional only checks "if arg_string in self._option_string_actions", that is, if the string can be found in the main parser actions.

So either this method needs to check the subparser actions, or the "if arg_string in self._option_string_actions" stuff needs to be delayed until the subparser. Neither of these seems like a simple change, but I agree it's a bug.

jakub · 2012-03-19T18:10:16Z

Attached quick&dirty fix I'm currently using in my project. Maybe this will help to create a valid patch.

Instead of changing _parse_optional, I've overridden _get_option_tuples to scan subparsers for matching optional arguments if option is ambiguous in the current parser.

I've only skimmed through the code, so forgive me if that fix is useless.

bethard · 2012-07-21T20:02:49Z

I think the real fix needs to search recursively though all subparsers. Here's a first draft of a patch that does this. I believe your sample code works after this patch.

I don't have the time to turn your bug report into proper test (and add a test for the recursive bit), but hopefully this is enough to let someone else finish up fixing this bug.

And yes, your "dirty hack" was still helpful in putting this together. ;-)

paulj3 · 2013-09-26T23:04:20Z

Steven's patch (subparse_optionals.diff) run with jakub's test case (argparse_subparses_ambiguous_bug.py) works. But if the input string is

    print(parser.parse_args('--foo baz'.split()))

produces

    Namespace(cmd=None, foo='baz', foo1=None, foo2=None)

(I added the 'cmd' subparse 'dest').

Two things seem to be going on now:

'--foo' is being parsed even though its subparser is not invoked,
and the subparser is not required.

The issue of whether subparsers are required or not is another issue. They used to be required, but the testing for 'required' was changed, and subparsers fell through the crack.

I suspect that if the missing subparser error were raised, the first issue wouldn't be apparent.

paulj3 · 2013-09-28T06:32:35Z

I think the correction to the problem that I noted in the previous post is to return 'None, arg_string, None', rather than 'action, arg_string, None' in the case where the action is found in a subparser.

This is what '_parse_optional' does at the end:

        # it was meant to be an optional but there is no such option
        # in this parser (though it might be a valid option in a subparser)
        return None, arg_string, None

An input like '--foo baz' would then produce an 'invalid choice' error. Since '--foo' is an optional that the primary parser does not recognize, 'baz' in interpreted as a positional, in this case an invalid subparser choice.

I'm working on cleaning up a test script.

paulj3 · 2013-09-28T06:35:16Z

In the last patch, 'parser.scan = True' is needed to activate this fix. I left that switch in for testing convenience.

paulj3 · 2013-09-29T04:30:28Z

This the argparse patch as described in the previous post, along with test_argparse tests.

For now the tests handle both subparser required cases and not required ones ( http://bugs.python.org/issue9253 ). While error messages can differ based on this requirement, it does not affect the 'ambiguity' that underlies the current issue.

paulj3 · 2014-08-07T22:47:32Z

Another issue dealing with abbreviations is close to being committed.

http://bugs.python.org/issue14910
argparse: disable abbreviation

Conchylicultor · 2022-09-27T14:18:48Z

It's 2022 and I'm still affected by this in Python 3.10.

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--flagA", type=str)
parser.add_argument("--flagB", type=str)
subparser = parser.add_subparsers(title="command")
build_parser = subparser.add_parser("build")
build_parser.add_argument("--flag", type=str)

parser.parse_args(["build", "--flag=asd"])

Raises

debug_flags.py: error: ambiguous option: --flag=asd could match --flagA, --flagB

The fix is to set allow_abbrev=False:

parser = argparse.ArgumentParser(allow_abbrev=False)

…(comment)) PiperOrigin-RevId: 477165673

…(comment)) PiperOrigin-RevId: 477169487

… parent parser and subparsers in argparse Check for ambiguous options if the option is consumed, not when it is parsed.

serhiy-storchaka · 2024-09-26T19:28:28Z

#124631 fixes this issue by delaying the check for ambiguous options until the option been consumed. The option for subparser is consumed by the subparser, not the parent parser, so it is no longer error.

…t parser and subparsers in argparse (GH-124631) Check for ambiguous options if the option is consumed, not when it is parsed.

… parent parser and subparsers in argparse (pythonGH-124631) Check for ambiguous options if the option is consumed, not when it is parsed. (cherry picked from commit 3f27153) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

… in the parent parser and subparsers in argparse (pythonGH-124631) Check for ambiguous options if the option is consumed, not when it is parsed. (cherry picked from commit 3f27153) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…e parent parser and subparsers in argparse (GH-124631) (GH-124759) Check for ambiguous options if the option is consumed, not when it is parsed. (cherry picked from commit 3f27153) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

In Python 3.13 and 3.12.7, the behavior of _parse_optional has changed. It used to raise an error on multiple matching actions itself, and only ever return None or an option tuple. Now the "raise error on multiple matching actions" code was moved out into consume_optional, and _parse_optional returns either None or a *list* of option tuples, which contains more than one if multiple actions match. See: python/cpython#124631 python/cpython#58573 This adapts to the change in a way that should work on both older and newer Pythons. I re-implemented the handling of multiple matching options so we don't have to worry about licensing and attribution if we copied in upstream's version. Signed-off-by: Adam Williamson <awilliam@redhat.com>

In Python 3.13 and 3.12.7, the behavior of _parse_optional has changed. It used to raise an error on multiple matching actions itself, and only ever return None or an option tuple. Now the "raise error on multiple matching actions" code was moved out into consume_optional, and _parse_optional returns either None or a *list* of option tuples, which contains more than one if multiple actions match. See: python/cpython#124631 python/cpython#58573 This adapts to the change in a way that should work on both older and newer Pythons. Signed-off-by: Adam Williamson <awilliam@redhat.com>

In Python 3.13 and 3.12.7, the behavior of _parse_optional has changed. It used to raise an error on multiple matching actions itself, and only ever return None or an option tuple. Now the "raise error on multiple matching actions" code was moved out into consume_optional, and _parse_optional returns either None or a *list* of option tuples, which contains more than one if multiple actions match. See: python/cpython#124631 python/cpython#58573 This adapts to the change in a way that should work on both older and newer Pythons. NOTE: This includes the fix for the typo that is in a separate commit on master. Signed-off-by: Adam Williamson <awilliam@redhat.com> Signed-off-by: Brian C. Lane <bcl@redhat.com> Resolves: RHEL-61426

…e parent parser and subparsers in argparse (GH-124631) (GH-124760) Check for ambiguous options if the option is consumed, not when it is parsed. (cherry picked from commit 3f27153)

This was a regression introduced in pythongh-58573. It was only tested for the case when the ambiguous option is the last argument in the command line.

…5273) This was a regression introduced in gh-58573. It was only tested for the case when the ambiguous option is the last argument in the command line.

…ythonGH-125273) This was a regression introduced in pythongh-58573. It was only tested for the case when the ambiguous option is the last argument in the command line. (cherry picked from commit 63cf4e9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…GH-125273) (GH-125360) This was a regression introduced in gh-58573. It was only tested for the case when the ambiguous option is the last argument in the command line. (cherry picked from commit 63cf4e9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…GH-125273) (GH-125359) This was a regression introduced in gh-58573. It was only tested for the case when the ambiguous option is the last argument in the command line. (cherry picked from commit 63cf4e9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

jakub mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 18, 2012

bethard mannequin closed this as completed Mar 18, 2012

bethard mannequin reopened this Mar 19, 2012

ezio-melotti transferred this issue from another repository Apr 10, 2022

erlend-aasland added this to Argparse issues May 19, 2022

erlend-aasland moved this to Bugs in Argparse issues May 19, 2022

Conchylicultor mentioned this issue Sep 27, 2022

argparse arguments in main parser hide an argument in subparser #73963

Closed

copybara-service bot mentioned this issue Sep 27, 2022

Fix flag conflicts caused by argparse (context: https://github.com/python/cpython/issues/58573#issuecomment-1259579063) tensorflow/datasets#4170

Closed

copybara-service bot pushed a commit to tensorflow/datasets that referenced this issue Sep 27, 2022

Fix flag conflicts caused by argparse (context: python/cpython#58573 …

779387d

…(comment)) PiperOrigin-RevId: 477165673

copybara-service bot pushed a commit to tensorflow/datasets that referenced this issue Sep 27, 2022

Fix flag conflicts caused by argparse (context: python/cpython#58573 …

4854e55

…(comment)) PiperOrigin-RevId: 477169487

bedevere-app bot mentioned this issue Sep 26, 2024

gh-58573: Fix conflicts between abbreviated long options in the parent parser and subparsers in argparse #124631

Merged

serhiy-storchaka self-assigned this Sep 26, 2024

serhiy-storchaka added 3.12 only security fixes 3.13 bugs and security fixes 3.14 bugs and security fixes labels Sep 26, 2024

serhiy-storchaka mentioned this issue Sep 26, 2024

argparse: ambiguous option reported with primary parser #109990

Closed

serhiy-storchaka added a commit that referenced this issue Sep 29, 2024

gh-58573: Fix conflicts between abbreviated long options in the paren…

3f27153

…t parser and subparsers in argparse (GH-124631) Check for ambiguous options if the option is consumed, not when it is parsed.

bedevere-app bot mentioned this issue Sep 29, 2024

[3.12] gh-58573: Fix conflicts between abbreviated long options in the parent parser and subparsers in argparse (GH-124631) #124759

Merged

bedevere-app bot mentioned this issue Sep 29, 2024

[3.13] gh-58573: Fix conflicts between abbreviated long options in the parent parser and subparsers in argparse (GH-124631) #124760

Merged

serhiy-storchaka closed this as completed Sep 29, 2024

github-project-automation bot moved this from Bugs to Doc issues in Argparse issues Sep 29, 2024

AdamWill mentioned this issue Oct 2, 2024

options: adjust to behavior change in upstream _parse_optional pykickstart/pykickstart#508

Merged

This was referenced Oct 10, 2024

gh-125254: Fix error report about ambiguous option in argparse #125273

Merged

Argparse logs incorrect ambiguous option #125254

Closed

miss-islington mentioned this issue Oct 12, 2024

[3.13] gh-125254: Fix error report about ambiguous option in argparse (GH-125273) #125359

Merged

miss-islington mentioned this issue Oct 12, 2024

[3.12] gh-125254: Fix error report about ambiguous option in argparse (GH-125273) #125360

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

argparse: subparsers, argument abbreviations and ambiguous option #58573

argparse: subparsers, argument abbreviations and ambiguous option #58573

jakub mannequin commented Mar 18, 2012 •

edited by bedevere-app bot

Loading

jakub mannequin commented Mar 18, 2012

tshepang mannequin commented Mar 18, 2012

tshepang mannequin commented Mar 18, 2012

bethard mannequin commented Mar 18, 2012

jakub mannequin commented Mar 19, 2012

bethard mannequin commented Mar 19, 2012

bethard mannequin commented Mar 19, 2012

jakub mannequin commented Mar 19, 2012

bethard mannequin commented Jul 21, 2012

paulj3 mannequin commented Sep 26, 2013

paulj3 mannequin commented Sep 28, 2013

paulj3 mannequin commented Sep 28, 2013

paulj3 mannequin commented Sep 29, 2013

paulj3 mannequin commented Aug 7, 2014

Conchylicultor commented Sep 27, 2022 •

edited

Loading

serhiy-storchaka commented Sep 26, 2024

argparse: subparsers, argument abbreviations and ambiguous option #58573

argparse: subparsers, argument abbreviations and ambiguous option #58573

Comments

jakub mannequin commented Mar 18, 2012 • edited by bedevere-app bot Loading

Linked PRs

jakub mannequin commented Mar 18, 2012

tshepang mannequin commented Mar 18, 2012

tshepang mannequin commented Mar 18, 2012

bethard mannequin commented Mar 18, 2012

jakub mannequin commented Mar 19, 2012

bethard mannequin commented Mar 19, 2012

bethard mannequin commented Mar 19, 2012

jakub mannequin commented Mar 19, 2012

bethard mannequin commented Jul 21, 2012

paulj3 mannequin commented Sep 26, 2013

paulj3 mannequin commented Sep 28, 2013

paulj3 mannequin commented Sep 28, 2013

paulj3 mannequin commented Sep 29, 2013

paulj3 mannequin commented Aug 7, 2014

Conchylicultor commented Sep 27, 2022 • edited Loading

serhiy-storchaka commented Sep 26, 2024

jakub mannequin commented Mar 18, 2012 •

edited by bedevere-app bot

Loading

Conchylicultor commented Sep 27, 2022 •

edited

Loading