Skip to content

Iterating over enum.Flag types ignores zero-valued in Python 3.11 #109633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gmgunter opened this issue Sep 21, 2023 · 10 comments
Closed

Iterating over enum.Flag types ignores zero-valued in Python 3.11 #109633

gmgunter opened this issue Sep 21, 2023 · 10 comments
Assignees
@ethanfurman
Labels
type-bug An unexpected behavior, bug, or error

Comments

@gmgunter
Copy link

gmgunter commented Sep 21, 2023 •
edited by github-actions bot
Loading

Bug report

Bug description:

In Python 3.10, you can iterate over enum.Enum and enum.Flag types to yield their values.

>>> import sys; print(sys.version)
3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]

>>> import enum
>>> class MyEnum(enum.Enum):
...     A = 0
...     B = 1
...     C = 2
...
>>> list(MyEnum)
[<MyEnum.A: 0>, <MyEnum.B: 1>, <MyEnum.C: 2>]

>>> class MyFlag(enum.Flag):
...     A = 0
...     B = 1
...     C = 2
...
>>> list(MyFlag)
[<MyFlag.A: 0>, <MyFlag.B: 1>, <MyFlag.C: 2>]

In Python 3.11, the same is true for enum.Enum types, but the behavior seems to have changed for enum.Flag types. Now, iterating over the type only yields its nonzero values.

>>> import sys; print(sys.version)
3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0]

>>> import enum
>>> class MyEnum(enum.Enum):
...     A = 0
...     B = 1
...     C = 2
...
>>> list(MyEnum)
[<MyEnum.A: 0>, <MyEnum.B: 1>, <MyEnum.C: 2>]

>>> class MyFlag(enum.Flag):
...     A = 0
...     B = 1
...     C = 2
...
>>> list(MyFlag)
[<MyFlag.B: 1>, <MyFlag.C: 2>]  # Missing <MyFlag.A: 0> (!)

CPython versions tested on:

3.10, 3.11

Operating systems tested on:

Linux
@gmgunter gmgunter added the type-bug An unexpected behavior, bug, or error label Sep 21, 2023
@CharlieZhao95
Copy link
Contributor

I bisected to here:

commit 7aaeb2a
Author: Ethan Furman ethan@stoneleaf.us
Date: Mon Jan 25 14:26:19 2021 -0800
bpo-38250: [Enum] single-bit flags are canonical (GH-24215)

@sobolevn
Copy link
Member

From the docs, I can say that now 0 is treated as more like a special value:
https://docs.python.org/3/library/enum.html#enum.Flag

Check how <Color: 0> is used all over the place.

The technical reason behind this is that now 0 does not get into _member_names_ here:

cpython/Lib/enum.py

Lines 313 to 319 in 712cb17

elif (
Flag is not None
and issubclass(enum_class, Flag)
and _is_single_bit(value)
):
# no other instances found, record this member in _member_names_
enum_class._member_names_.append(member_name)

Because 0 is not a _is_single_bit member (it is explicit):

cpython/Lib/enum.py

Lines 93 to 98 in 712cb17

def _is_single_bit(num):
"""
True if only one bit set in num (should be an int)
"""
if num == 0:
return False

So, when we use __iter__ over _member_names_ zero-value is not there:

cpython/Lib/enum.py

Lines 790 to 794 in 712cb17

def __iter__(cls):
"""
Return members in definition order.
"""
return (cls._member_map_[name] for name in cls._member_names_)

We need @ethanfurman :)

@ethanfurman ethanfurman self-assigned this Sep 21, 2023
@ethanfurman
Copy link
Member

It is more accurate to say that Enum and Flag iterate over the canonical values. For example:

class Color(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3
    ROJO = 1
    VERDE = 2
    AZUL = 3

list(Color)
# RED, GREEN, BLUE

class BlendColor(Flag):
    BLACK = 0
    RED = 1
    GREEN = 2
    BLUE = 4
    WHITE = RED | GREEN | BLUE

list(BlendColor)
# RED, GREEN, BLUE

Aliases have always been omitted from iteration, and flag values with no, or more than one, bit(s) set are considered aliases since Python 3.11 -- this was considered a bug-fix.

@gmgunter
Copy link
Author

Oh I see. Thanks for the clarification, @ethanfurman.

Looking back over the docs now, they do pretty much convey everything that you've explained here, although some of the details are subtle. The only thing that I think isn't explicitly explained there is that zero-valued flags are considered aliases.

@bindreams bindreams mentioned this issue May 1, 2024
Open
@wimglenn
Copy link
Contributor 

>>> class F(enum.Flag):
...     f = 0
... 
>>> len(F)
0
>>> list(F)
[]

Why isn't F.f considered to be canonical?

@ethanfurman
Copy link
Member

Because f is not a flag, it is a name for the absence of flags.

@wimglenn
Copy link
Contributor

wimglenn commented Jun 27, 2024 •
edited
Loading

f shows up in F.__members__, but not in dir(F). Is that intended?

@ethanfurman
Copy link
Member

Yes. dir(F) doesn't list aliases, but __members__ does (which also prevents aliases from being overridden -- i.e. F.f = 99 raises an AttributeError).

@ClundXIII
Copy link

Hi,

I do have a use case where I need both alias and elements of a flag. Is there a way to explicitly iterate over either only elements and only aliases that works in both old AND new version?

@Leo1690
Copy link

Leo1690 commented Mar 21, 2025

@ClundXIII You can do this

from enum import IntFlag

class Bla(IntFlag):
    X = 0

list(Bla.__members__.values())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ethanfurman ethanfurman

Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@ClundXIII @sobolevn @wimglenn @ethanfurman @gmgunter @Leo1690 @CharlieZhao95