Exprimental WASI support for ports/unix #13676

yamt · 2024-02-15T07:44:36Z

See comments in build-wasi.sh

codecov · 2024-02-15T07:47:12Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.57%. Comparing base (bdda91f) to head (5098523).

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #13676   +/-   ##
=======================================
  Coverage   98.57%   98.57%           
=======================================
  Files         164      164           
  Lines       21349    21349           
=======================================
  Hits        21045    21045           
  Misses        304      304

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

stinos · 2024-02-15T07:51:55Z

ports/unix/mpconfigport.h

@@ -154,8 +154,12 @@ typedef long mp_off_t;
 // Don't default sys.argv and sys.path because we do that in main.
 #define MICROPY_PY_SYS_PATH_ARGV_DEFAULTS (0)

+#if defined(__wasi__)


Should be done like

#ifndef MICROPY_PY_SYS_EXECUTABLE #define MICROPY_PY_SYS_EXECUTABLE (1) #endif

so it works generically, then make a separate mpconfigvariant_wasi.h where it gets set to 0.

who would include mpconfigvariant_wasi.h?

That would be done automatically if you use make VARIANT=wasi (though, my mistake, it should actually be unix/variants/wasi/mpconfigvariant.h). Likewise instead of build.sh you'd implement the specific makefile options in unix/variants/wasi/mpconfigvariant.mk. Check out the other variants for examples. Also I think to have any chance of merging this ulab should probably not be included unconditionally because this makes it harder to build out of the box; on the other hand if this only works with non-approved upstream wasi patches it's not super useful for others anyway.

Yes, please make this a new variant. It should then be possible to build it via make VARIANT=wasi.

ok. i was not aware of the mechanism. thank you. let me try to adapt.

Also I think to have any chance of merging this ulab should probably not be included unconditionally because this makes it harder to build out of the box; on the other hand if this only works with non-approved upstream wasi patches it's not super useful for others anyway.

sure.

wrt wasi patches, alternatively it's possible to put the setjmp/longjmp runtime within micropython itself. at least until when setjmp/longjmp gets ready in wasi. how do you think?

anyway, it won't be super useful until wasm EH is widely implemented in runtimes though.

If building unix for wasi requires patches to wasi/binaryen then IMO this PR should stay as a draft until those patches are no longer needed. You could put the setjmp/longjmp code here, but I guess the other patches are still needed.

i adapted to use VARIANT. it simplify a lot. thank you.

i'm not sure what should be done for UNAME_S logic in Makefile though.
a build for WASI is actually a cross compile. but the existing CROSS_COMPILE seems geared to "arm-none-eabi-gcc" style cross toolchain, which doesn't seem suitable to wasi-sdk style cross toolchain.

If building unix for wasi requires patches to wasi/binaryen then IMO this PR should stay as a draft until those patches are no longer needed. You could put the setjmp/longjmp code here, but I guess the other patches are still needed.

FYI, all toolchain patches have been merged in the upstream since then.

stinos · 2024-02-15T07:52:40Z

ports/unix/variants/mpconfigvariant_common.h

@@ -91,7 +91,11 @@
 #define MICROPY_PY_OS_ERRNO            (1)
 #define MICROPY_PY_OS_GETENV_PUTENV_UNSETENV (1)
 #define MICROPY_PY_OS_SEP              (1)
+#if defined(__wasi__)


Similar comment as above: instead of modifying this variant it might be better to create it separately

dpgeorge · 2024-02-16T01:39:33Z

See semi-related #11723, which gets mpy-cross compiling to WASM (but not WASI at this stage).

github-actions · 2024-05-02T06:50:48Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

yamt · 2024-07-05T09:17:21Z

is there any TODO for this PR while waiting for LLVM 19?

yamt · 2024-07-26T08:22:14Z

the last push is a conflict resolution