Skip to content

dict_set_fromkeys() calculates size of dictionary improperly #132762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ThomasBr0 opened this issue Apr 21, 2025 · 2 comments
Open

dict_set_fromkeys() calculates size of dictionary improperly #132762

ThomasBr0 opened this issue Apr 21, 2025 · 2 comments
Assignees
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@ThomasBr0
Copy link

ThomasBr0 commented Apr 21, 2025

Bug report

Bug description:

The function dict_set_fromkeys() in the file dictobject.c adds elements of an iterable to an existing dictionary. The size of the expanded dictionary is estimated as PySet_GET_SIZE(iterable) and the size of the existing dictionary is not considered.

This is unlogical.

A more resonable estimation is "mp->ma_used + PySet_GET_SIZE(iterable)". What turns this into a bug is the fact that a large dictionary plus a small iterable can cause the function to loop forever.

The following code is adopted from the official tests, file test_dict.py. Official binaries for Python 3.13.3 compiled with MSVC will loop forever when running this modified test (the official test has range(10)). Other platforms may be affected as well.

class baddict3(dict):
    def __new__(cls):
        return d
d = {i : i for i in range(17)}
res = d.copy()
res.update(a=None, b=None, c=None)
print(res)
q = baddict3.fromkeys({"a", "b", "c"})
print(q)

A minor error is in the function calculate_log2_keysize(minsize) which has three branches for calculating its result. However, when minsize < 10 the results of the three branches can differ. Since _BitScanReverse64() is Windows-only other platforms might be unaffected.

A final unrelated note. PEP 11 – CPython platform support states that aarch64-unknown-linux-gnu and x86_64-unknown-linux-gnu are Tier 1 AND Tier 2.

CPython versions tested on:

3.13

Operating systems tested on:

Windows

@ThomasBr0 ThomasBr0 added the type-bug An unexpected behavior, bug, or error label Apr 21, 2025
@ZeroIntensity ZeroIntensity added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Apr 21, 2025
@ZeroIntensity
Copy link
Member

Looks like a 3.13 regression. On a debug build, this assertion fails:

python: Objects/dictobject.c:2025: dictresize: Assertion `newkeys->dk_usable >= mp->ma_used' failed.

I'm pretty sure the regressing PR was gh-114508 (cc @DinoV @colesbury).

@ZeroIntensity ZeroIntensity added 3.13 bugs and security fixes 3.14 new features, bugs and security fixes labels Apr 21, 2025
@colesbury
Copy link
Contributor

@DinoV - would you please take a look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants