-
-
Notifications
You must be signed in to change notification settings - Fork 8.2k
Core implementation of OrderedDict #1162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I initially wanted to subclass dict and override setter methods, but then found this opportunity to (ab)use table_is_fixed_array flag. Note that I didn't even bother to introduce another flag. Semantics of that flag might have had connotation of "stored in ROM", but then I'd want to redefine it right away. Because "R/O data structure" is a bigger problem (we once touched it), and solving it just for maps doesn't make sense. That's also the reason why I didn't ifdef new additions to map.c. Another reason is that they might be useful even w/o explicit exposing of OrderedDict (for example, there was talk on python-dev on preserving kwarg order). Note that subclassing native types on its own doesn't work, #1159 |
As can be seen, this breaks cmd_showbc & cmd_verbose. What can I say - I understand the motives why they were added - to get coverage to a magic number of 90%. But they surely are fragile. So, I didn't look into them, hoping that you, @dpgeorge, already worked around bunch of cases, and can spot what went wrong again more easily than I ;-). At the very least, .out file should be written with masked fields, so it was easy to diff real, unmasked differences. |
It's neat that you can reuse this flag. BUT, it is very important that this flag means "sored in ROM" and that you can't change the dict. Eg with your patch the following now crashes: list.a = 1 All types and modules use a constant map/dict to implement their lookup tables. It's nothing to do with const data from a bytecode point of view (ie #573). |
Well, but the same may apply to e.g. lists. I can't give immediate example (sys.path = 1 somehow doesn't crash), but that's still the issue. I proposed to have is_ro(void_) or is_rw(void_) as generic solution (would need good thought which is better - they become non-trivial in the presence of loadable modules). Of course, if you want to still postpone it and rather have me still another bit from map used size (or maybe start to steal bits from alloc size to keep it balanced), ok. |
A constant list is a tuple. There are no lists that are in ROM (as far as I know). |
Just to clarify motivation for this patch: uctypes uses dict as the most memory-efficient structure for the data. But calculating structure field offsets "on the fly" (well, not by user) requires ordered dict. And without calculating offsets by uctypes itself, almost any uctypes description for Unix is arch-dependent, which renders it unusable in real life. |
Given that there's already support for "fixed table" maps, which are essentially ordered maps, the implementation of OrderedDict just extends "fixed table" maps with add/remove operations, and reuses 95% of objdict code, just making methods tolerant to both dict and OrderedDict. Some things are missing so far, like CPython-compatible repr and comparison.
Mostly to have coverage of newly added code in map.c.
Ok, added separate flag for ordered maps. Left it in the same word as fixed to optimize || testing. all_keys_are_qstrs would be better candidate for moving, but left for later. |
Sure, I added separate commit to make changes more visible and easier to review, intended to squash it either once ok'ed. |
And, now the biggy question - to make ordered dicts really efficient way to deal with data, it would be nice to support literal ordered dicts. Do you think you could change compiler to recognize OrderedDict "keyword" and create an ordered dict? ;-) |
Oh my, no, you couldn't, because we don't support dict literals at all :-(. |
That's because they are mutable. We need frozendict first, then we can have FrozenOrderedDict. Maybe upstream (ie CPython) can be convinced to add frozendict?? :) |
There's kind of read-only dict type in CPython:
But what we really need, as we discussed once, a way to have dict literals, to either serve as a template to create a new dict instance, or if it can be proven that it can be used only once, to be used as object directly. But that still has bytecode persistence on critical path to be done right. |
No description provided.