Skip to content

re.sub appears not to check count optional argument for integerness #73091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
DannyYoo mannequin opened this issue Dec 8, 2016 · 4 comments
Closed

re.sub appears not to check count optional argument for integerness #73091

DannyYoo mannequin opened this issue Dec 8, 2016 · 4 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@DannyYoo
Copy link
Mannequin

DannyYoo mannequin commented Dec 8, 2016

BPO 28905
Nosy @bitdancer
Superseder
  • bpo-28082: re: convert re flags to (much friendlier) IntFlag constants
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-12-08.18:37:08.796>
    created_at = <Date 2016-12-08.17:02:58.334>
    labels = ['type-bug', 'library']
    title = 're.sub appears not to check count optional argument for integerness'
    updated_at = <Date 2016-12-08.18:37:08.794>
    user = 'https://bugs.python.org/DannyYoo'

    bugs.python.org fields:

    activity = <Date 2016-12-08.18:37:08.794>
    actor = 'r.david.murray'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-12-08.18:37:08.796>
    closer = 'r.david.murray'
    components = ['Library (Lib)']
    creation = <Date 2016-12-08.17:02:58.334>
    creator = 'Danny Yoo'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 28905
    keywords = []
    message_count = 4.0
    messages = ['282719', '282721', '282722', '282726']
    nosy_count = 2.0
    nosy_names = ['r.david.murray', 'Danny Yoo']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '28082'
    type = 'behavior'
    url = 'https://bugs.python.org/issue28905'
    versions = ['Python 3.4']

    @DannyYoo
    Copy link
    Mannequin Author

    DannyYoo mannequin commented Dec 8, 2016

    This comes from diagnosing a beginner's question on Python-tutor.

    https://mail.python.org/pipermail/tutor/2016-December/110066.html

    It appears that re.sub is not checking whether the count argument is integer or not, and silently accepts a nonsensical argument. For example:

    >>> import re
    >>> s = "AAAcBBB\nAAAdBBB"
    >>> print(re.sub(r'^AAA', "aaa", s, re.MULTILINE))
    aaacBBB
    AAAdBBB

    Of course, the user intended to pass re.MULTILINE to flags, not to count, but the fact that this isn't raising a TypeError is error-prone.

    @DannyYoo DannyYoo mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Dec 8, 2016
    @DannyYoo
    Copy link
    Mannequin Author

    DannyYoo mannequin commented Dec 8, 2016

    Ugh. I suddenly realize that this is complicated by the fact that flag values are themselves represented as integers, and Python's type system isn't rich enough to label flag values as a distinct type for the purposes.

    It may be worthwhile to add a warning in the documentation about this, as it is an easy mistake to make.

    @DannyYoo
    Copy link
    Mannequin Author

    DannyYoo mannequin commented Dec 8, 2016

    Alternatively, change the representation of flag values from integers to some class extension that supports the common bitwise operators.

    As a very rough sketch:

    >>> class FlagInt(int):
    ...     def __or__(self, other):
    ...         return FlagInt(int(self) | int(other))
    ... 
    >>> f1 = FlagInt(1)
    >>> f2 = FlagInt(2)
    >>> f1 | f2
    3
    >>> isinstance(3, FlagInt)
    False
    >>> isinstance(f1 | f2, FlagInt)
    True

    That way, flag arguments can be determined at runtime to have derived from the proper flag values.

    This kind of approach may have some backwards-incompatibility, unfortunately, since other folks have been hardcoding integers rather than use the flag constants. Other concerns might include serialization, in case someone tries to save a FlagInt somewhere and pull it out at some other time.

    @bitdancer
    Copy link
    Member

    See bpo-28082.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant