Skip to content

unix: struct/ustruct not consistent with pyboard #1740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dhylands opened this issue Dec 22, 2015 · 11 comments
Closed

unix: struct/ustruct not consistent with pyboard #1740

dhylands opened this issue Dec 22, 2015 · 11 comments

Comments

@dhylands
Copy link
Contributor

On the pyboard, import struct works.

On micropython, import ustruct works, but import struct doesn't (assuming that you haven't installed micropython-lib/struct).

Shouldn't struct be a weak-link to ustruct? (as it is on the pyboard?)

What about the other umodules?

@dpgeorge
Copy link
Member

I think the idea was that weak links can be made on the PC by making a file, eg struct.py, containing from ustruct import *. Well, that's not very convenient :)

We could make pyboard consistent with unix by removing weak links, but that's not the right approach here.

@dhylands
Copy link
Contributor Author

I was hoping that the solution would be to add the weak links to the unix version. But I can live with having to install a bunch of inconvenient files. I definitely wouldn't want to remove the weak links on the pyboard side.

@dpgeorge
Copy link
Member

I was hoping that the solution would be to add the weak links to the unix version.

I would consider this, I don't see any real downside. @pfalcon probably has an opinion here :)

@pfalcon
Copy link
Contributor

pfalcon commented Dec 23, 2015

Yes, I have, and it's indeed would be in direction that it's not unix port should be consistent with pyboard, but vice-versa, pyboard should be consistent with unix, which is in turn should be consistent with CPython. CPython doesn't have any "module symlinks", and that's good reason to not have them either. If some baremetal ports due to their resource/interaction limitations find them useful, ok, but I don't see a reason to consider them a generally useful feature (and whoever thinks otherwise, should just go to python-ideas or python-dev and convince CPython developers to implement them, so easy).

I'd elaborate more on this answer when I have more time. It would go along the lines that again, not unix port should be more consistent with pyboard, but vice versa, pyboard should acquire features of well-developed unix port. For example, on unix, installing "struct" module is as trivial as "micropython -m upip micropython-struct". And I'm surprised nobody works on making upip run on pyboard et al.

@dhylands
Copy link
Contributor Author

Not everybody wants to emulate CPython on bare-metal. Personally, I find the weak symlink feature on the bare metal ports to be extremely useful, and definitely don't want to see it ever go away, not matter how "pure" it makes things.

@pfalcon
Copy link
Contributor

pfalcon commented Dec 23, 2015

Nobody talks about removing it from baremetal ports - they may simply lack enough diskspace to handle it in normal "from uXXX import *" way.

But if you're sure then this feature should be added to unix port, then please reopen this ticket and let's collect more grounded arguments from many people during next half-year and then revisit this (those should be grounded opinions, not just random people casting their one-word votes - we suffer results of the latter re: very-very long executable name, which propagated to other names too, like very-very long PyPI package names, etc.)

@pfalcon
Copy link
Contributor

pfalcon commented Dec 23, 2015

Otherwise, the idea is:

If you write your app to run on bare uPy, use "import ustruct". If you write your app to be compatible with CPython/unix uPy/baremetal uPy, use "import struct" (unix uPy must have that module installed of course). If you happen to write your app to be compatible with bare uPy and CPython (that happens too), use:

try:
    import ustruct
except ImportError:
    import struct as ustruct

@dhylands
Copy link
Contributor Author

I'd rather no clutter up my code with try/except (which wastes memory). What I really want is to write code which works on bare-metal and CPython. So I'll either ignore the unix port since it won't work for me, or I'll add dummy modules. Since adding the dummy modules is inconvenient, I'll probably just ignore the unix port of micropython for this particular exercise.

@pfalcon
Copy link
Contributor

pfalcon commented Dec 23, 2015

What I really want is to write code which works on bare-metal and CPython. So I'll either ignore the unix

If you write it like that, it'll automagically work on unix port - as long as the standard library is installed. And it's normal way to use unix port - with the full standard library installed (just like CPython). It's just unix port allows to install fine-grained subset of stdlib, unlike CPython - that's important and high-priority usecase, but it's still a "special" usecase, the "normal" usecase is to have full micropython-lib installed.

@dhylands
Copy link
Contributor Author

Perhaps "normal" for you. The "normal" way for me is to not have the full standard library installed. I just run ~/micropyton/unix/micropython, exactly as it gets built from using the makefile. It does not "automagically" work as you put it.

Maybe once you get site.py integrarted and that can add a sys.path which points to the micropython-lib then it will automagically work.

@pfalcon
Copy link
Contributor

pfalcon commented Dec 23, 2015

Next step after building micropython is issuing "./micropython -m upip install micropython-all", or whatever name is selected in micropython/micropython-lib#47 , once it catches enough attention from the people to be implemented (I indeed have it installed even without it). But Python is all about libraries, and there's no way around that, we can't stuff everything into executable.

tannewt added a commit to tannewt/circuitpython that referenced this issue Apr 3, 2019
jimmo added a commit to jimmo/micropython that referenced this issue Oct 18, 2019
Previously ports chose whether or not to alias `ufoo` -> `foo`, and each
port implemented this explicitly. This changes objmodule to always provide a
`foo` weak link for any core `ufoo` module.

This should be a no-op on any ports currently enabling weak links.

`utime` and `uos` are provided by the ports, so they must still provide
their own weak links for these modules.

Fixes micropython#1740
dpgeorge added a commit that referenced this issue Oct 22, 2019
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: #1740, #4449, #5229, #5241.
alu96 pushed a commit to alu96/micropython that referenced this issue Mar 23, 2020
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: micropython#1740, micropython#4449, micropython#5229, micropython#5241.
c0d3z3r0 pushed a commit to c0d3z3r0/micropython that referenced this issue Apr 5, 2020
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: micropython#1740, micropython#4449, micropython#5229, micropython#5241.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants