Skip to content

MicroPython and emscripten #3474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LeaNumworks opened this issue Dec 7, 2017 · 15 comments
Closed

MicroPython and emscripten #3474

LeaNumworks opened this issue Dec 7, 2017 · 15 comments

Comments

@LeaNumworks
Copy link
Contributor

LeaNumworks commented Dec 7, 2017

Hi,
I work on a graphic calculator for the company Numworks. We have a Python app that uses MicroPython. We fixed some issues when using uPY on emscripten and successfully got it running.

Among the issues we found and fixed:

Would you be interested in some PRs to get a working Emscripten build?

(By the way, this could be interesting for issues #3313, #2618 , #1561, #888)

@dpgeorge
Copy link
Member

dpgeorge commented Dec 8, 2017

Hi @LeaNumworks, nice work with your simulator!

MicroPython assumes the stack is descending, which is not the case on emscripten.
We had trouble with garbage collection as explained in emscripten’s portability guidelines.

Myself and a colleague have also lately been working on an Emscripten port and came across these two issues as well. The first is easy to fix and we can provide a compile-time option in uPy to select the stack direction.

We could not fix the second issue yet, with the GC. We have a build of uPy that runs under node (eg node micropython.js <script.py>) and running this on the uPy test-suite leads to a fair few failures which are due to the GC not being able to trace root pointers. Even compiling with -O0 does not fix the issue. What was your approach to dealing with this problem?

Would you be interested in some PRs to get a working Emscripten build?

I'm definitely interested in having an Emscripten port, but the GC issue is a real show stopper at this point.

@LeaNumworks
Copy link
Contributor Author

I agree that having a working GC is a priority.

We only really did three things:

  • Fix the stack direction
  • Get the root pointers in gc_collect using setjmp
  • Compile uPy with -O0

We did not run the uPY test-suite on our port so we may indeed be missing something. That being said, it does a great job for our use-case and the few scripts we tried this on did not seem to be leaking memory.

@kkimdev
Copy link

kkimdev commented Dec 14, 2017

@LeaNumworks Would you mind sharing your branch?

In case fixing the garbage collector properly is a huge headache, how about having a naive implementation of https://github.com/micropython/micropython/blob/master/py/gc.c for Emscripten port for now and migrating to WebAssembly/design#1079 in the future when it's ready?

@kkimdev
Copy link

kkimdev commented Dec 14, 2017

@alanjds
Copy link

alanjds commented Dec 15, 2017

Nice!

I had put a bounty on #888 (comment) to get something like this done. Not that $100 USD could possibly cover the costs, but is more about an incentive to get it rolling.

Thanks very much for your effort.

@kkimdev
Copy link

kkimdev commented Dec 18, 2017

@LeaNumworks I'm trying to run uPy test suite with https://github.com/numworks/epsilon/tree/master/python , but looking at port/ , it looks like it has other dependencies like Ion. Is there an easy way to just run on desktop? I was thinking maybe I should use the default port/unix instead but I'm not that familiar with Micropython porting layer. Thanks.

@kkimdev
Copy link

kkimdev commented Dec 19, 2017

nvm I took some time to read the source code and have a better understanding of the structure now. I'll ask if I have any further question thanks!

@kkimdev
Copy link

kkimdev commented Jan 3, 2018

You can try @LeaNumworks 's stand-alone micropython here https://kkimdev.github.io/epsilon/index.html

repo: https://github.com/kkimdev/epsilon
I just stripped off non-micropython dependencies from https://github.com/numworks/epsilon/tree/master/python , it didn't require much work actually.

@alanjds
Copy link

alanjds commented Jan 3, 2018

181 KB download / 665 KB unpacked !

This is nice!

@dpgeorge
Copy link
Member

A port to javascript was added in 7d675f3

@GeorgeWort
Copy link

@LeaNumworks I am currently working on running MicroPython for the micro:bit with emscripten and emterpretify. I have applied your GC fixes, i.e. the descending stack, using setjmp, and I know that optimisations are not applied. Despite this, the GC still sweeps what it shouldn't, specifically, the local code_state pointer in fun_bc_call appears to not be stored in the stack but stored somewhere in emterpreter's local javascript heaps.

Am I right in thinking that you also use emterpretify?
If so, is there any reason you can think of that this might happen for me but not for the Numworks simulator?
i.e. How do you handle emterpretify storing pointers in heaps?
Is that the same issue you are trying to prevent with -O0?

@dpgeorge
Copy link
Member

@geowor01 you may want to test with #4723 to see if it helps your case

@GeorgeWort
Copy link

Thanks @dpgeorge! That does appear to fix the problem.

tannewt pushed a commit to tannewt/circuitpython that referenced this issue Sep 29, 2020
@jonnor
Copy link
Contributor

jonnor commented Jul 30, 2023

This issue can probably be closed now? The JavaScript port mentioned above has been renamed to WebAssembly, and should be generally functional. Any problems with it should probably be handled in specific issues?

@mattytrentini
Copy link
Contributor

Appears resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants