Skip to content

Implement automatic weak links for all built-in modules #5241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

dpgeorge
Copy link
Member

This is an alternative to #5229 which implements automatic module weak links for all built-in modules, by searching for "ufoo" in the built-in module list if "foo" cannot be found.

This means that all modules named "ufoo" are always available as "foo". Also, a port cannot add any other weak links (probably a good thing, to make weak links have a strict definition).

It also saves some code size (about 200 bytes) on ports that previously had lots of weak links.

Downsides (all minor IMO):

  • is probably a bit slower when searching due to using strcmp instead of comparing qstr values (but this is minor, and importing is anyway not a time-critical thing)
  • doesn't intern the non-u module names (eg "foo" is not interned), which saves code size, but will mean that "import foo" creates a new qstr (namely "foo") in RAM (unless the module is frozen, then it adds qstrs in ROM)
  • help('modules') no longer lists non-u module names, only the u-variants; this may be a good thing because it reduces duplication in this listing

Original idea from #4449 (comment)

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Oct 21, 2019
@jimmo
Copy link
Member

jimmo commented Oct 21, 2019

Looks good, I prefer this to #5229 -- thanks @dpgeorge !

@dpgeorge
Copy link
Member Author

Change in code size for this PR:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:  +168 +0.034% 
unix nanbox:  +228 +0.051% 
      stm32:  -228 -0.063% PYBV10
     cc3200:  -104 -0.056% 
    esp8266:  -188 -0.029% GENERIC
      esp32:  -173 -0.015% GENERIC[incl -208(data)]
        nrf:    -8 -0.005% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS

unix is up because it enables weak links, ports that already had weak links enabled are down.

dpgeorge added a commit that referenced this pull request Oct 22, 2019
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: #1740, #4449, #5229, #5241.
@dpgeorge
Copy link
Member Author

Merged in d2384ef and 1582c7e

@dpgeorge dpgeorge closed this Oct 22, 2019
@dpgeorge dpgeorge deleted the py-auto-weak-links branch October 22, 2019 05:25
alu96 pushed a commit to alu96/micropython that referenced this pull request Mar 23, 2020
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: micropython#1740, micropython#4449, micropython#5229, micropython#5241.
c0d3z3r0 pushed a commit to c0d3z3r0/micropython that referenced this pull request Apr 5, 2020
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: micropython#1740, micropython#4449, micropython#5229, micropython#5241.
tannewt pushed a commit to tannewt/circuitpython that referenced this pull request Aug 27, 2021
…-pybadge-b-m

Convert more modules to use MP_REGISTER_MODULE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants