This is an interesting concept. Did you happen to measure performance differences?

Did you happen to measure performance differences?

No, didn't get to that yet. I guess it wouldn't be too hard to do it for sum, just passing it in various things.

I did some benchmarking, running a simple loop like this:

def test():
    x = range(1000)
    for _ in range(300):
        sum(x)

Running on a PYBD-SF2 @ 120MHz I get:

diff of scores (higher is better)
N=100 M=100                   sum_c -> sum_bytecode         diff      diff% (error%)
builtin_sum_range.py        1023.28 ->     512.16 :    -511.12 = -49.949% (+/-0.14%)
builtin_sum_list.py         1106.45 ->     519.57 :    -586.88 = -53.042% (+/-0.20%)

So bytecode is half the speed of the existing C implementation.

Then I used mpy-cross to compile the Python implementation of sum to native Python code, and put that in instead of the frozen bytecode. The results for the benchmark of this native Python vs C were:

diff of scores (higher is better)
N=100 M=100                   sum_c -> sum_native         diff      diff% (error%)
builtin_sum_range.py        1023.28 ->     942.19 :     -81.09 =  -7.925% (+/-0.29%)
builtin_sum_list.py         1106.45 ->     998.18 :    -108.27 =  -9.785% (+/-0.22%)

That's not too bad, the native code generator is close to the C version!

See latest commit for the native code blob, for both x86-64 and Thumb2. The size of these blobs is a bit bigger than the C version, but with a bit of optimisation in the native emitter these blobs could become a bit smaller and faster, possibly getting close to C.

There is a lot of scope here for further work. As shown, it possible to reimplement parts of the core in Python, which is compiled to either bytecode (small but slow) or native machine code (fast but bigger, and potentially on par with the existing C implementation). A benefit of writing in Python rather than C is that the underlying architecture of the system (of MicroPython, the VM, etc) is hidden. For example, whether exceptions are implemented using NLR or simple return codes is irrelevant to the Python implementation of a function (eg sum()), and the underlying architecture can be more easily changed.

This is very meta and gets into the territory of PyPy (MicroPyPy!), where the interpreter is written in itself (RPython a reduced dialect of Python).

This is a very cool demonstration Damien. I'd like to see arguments, eventually, to both the make process for building MicroPython, and to mpy-cross to favour speed vs. size, that would set a MACRO that we can use throughout the code base to prefer for example Python vs C, like you have here.

Perhaps an option to favour less heap or more heap too?

I'd like to see arguments, eventually, to both the make process for building MicroPython, and to mpy-cross to favour speed vs. size, that would set a MACRO that we can use throughout the code base to prefer for example Python vs C, like you have here.

Yes, that makes sense, speed vs size (like -Os vs -O2).

Perhaps an option to favour less heap or more heap too?

It might be tricky to have that option orthogonal to speed-vs-size. In general the code always attempts to reduce heap usage.

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.

I will close this because it's not really going anywhere. Maybe it can be revisited one day.