Skip to content

Releases: micropython/micropython

I2CTarget, improved floats and native emitter, STM32N6 & ESP32C2 support

09 Aug 15:08
Compare
Choose a tag to compare

This release of MicroPython sees the introduction of machine.I2CTarget, which allows Python code to implement an I2C target device. It's available on the alif, esp32, mimxrt, rp2, samd, stm32 and zephyr ports. In the simplest case it can create an I2C register/memory device that connects to a bytearray (or similar buffer) and allows an I2C controller to read from and write into that bytearray. For more complex scenarios the I2CTarget class exposes a set of interrupts which can be acted upon by Python code, allowing an arbitrary I2C device to be created. See the documentation for more details and examples.

Floating point support has been improved in a few ways in this release. First, the formatting (printing) of floats has been rewritten to significantly improve the repr reversibility of floating-point numbers. That is, formatting and then re-parsing a float should return the original value. In MicroPython the percentage of floats that correctly repr and parse back was around 28% (single precision) and 38% (double precision), but has now been improved (in the standard build configuration) to 98.5% and 99.8% (single and double respectively). In addition to printing out more accurate floats, that helps a lot when saving floats to .mpy files, because it means they can be loaded back to the same value.

Second, there is now support in the compiler for handling floats as constants. Floats can now be folded (along with integers) in arithmetic operations, and be part of const assignments. This helps to optimize bytecode because constant float expressions can now be evaluated in the compiler instead of at runtime.

The third float improvement involves object representation C, where floats are stored within the immediate object value rather than on the heap. In this object module, the float can only take up 30 bits, so the last two bits of the single precision number are lost. Previously they were just truncated, but now a heuristic is applied when recovering the float value from the object such that the last two bits are copied from the previous two. This alleviates a bias towards zero and in general improves floating point calculations when using this representation.

The native and viper emitter backends have been improved to emit more optimal machine code, for example to use more compact instructions in loads and stores. This is for all supported architectures: ARM, Thumb, Xtensa, RISC-V 32, x86 and x64. Thumb v1 (for example RP2040) now supports long jumps greater than 12 bits, allowing larger Python functions to be compiled to this native architecture. Also, the inline Xtensa assembler now implements most of the LX3 opcodes, with additions including addx2, subx2, ssl and ssr, among others.

Two new MCUs are supported by this release: STM32N6xx and ESP32-C2 (aka ESP8684). The STM32N6xx is a new STMicroelectronics MCU running at 800MHz with a large amount of RAM and machine-learning accelerators. MicroPython now supports this MCU with USB, XSPI memory-mapped external flash, a filesystem, basic peripherals and deepsleep. The ESP32-C2 is by Espressif and is a small and low cost RISC-V MCU with WiFi and BLE. MicroPython running on this MCU supports a REPL, filesystem, GPIO, I2C, ADC, PWM, timers, WiFi and BLE.

The date range supported by the time module has now been standardized to work the same across all platforms. The functions time(), localtime() and mktime() now work properly on a reasonable range of dates, from at least 1970 up to 2099, regardless of the Epoch used by the platform.

The MicroPython virtual machine is now able to avoid heap-allocating slices when subscripting bytearray and memoryview objects, for example expressions like bytearray_obj[a:b] = c. The slice object is now allocated on the C stack, helping to reduce memory churn and allowing such expressions to work when the heap is locked, for example during a hard interrupt handler. This improvement goes in the general direction of MicroPython using the heap as little as possible, making execution more deterministic.

Other improvements to the core runtime include support for using __all__ in star imports, support for PEP487's __set_name__ special method, support for the _b/o/x specifier in str.format, and arrays can now be extended from any iterable, not just from an object with the buffer protocol. A new MicroPython-specific sys.implementation._thread attribute has been added, which exists when threading is enabled and tells which threading model is used (GIL or unsafe/no-GIL).

The framebuf module now supports blit'ing read-only data to a FrameBuffer, which helps when implementing custom fonts that can now be stored in ROM. The DTLS implementation in the tls module now enables DTLS HelloVerify and Anti Replay protection (for the mbedTLS backend), allowing proper DTLS servers to be implemented. The lwIP socket layer now has a queue of incoming UDP and raw packets (previously it only had room for one outstanding packet), allowing for more robust and efficient UDP protocols.

Mpremote has had a few improvements. There is a new fs tree command which mimics the Unix tree command, and includes the -s and -h options to show the file sizes in the output. The df command has been enhanced to use the new no-argument vfs.mount() query, and shows a much better summary of the mounted filesystems. The mip install command now uses hashes to skip files that exist. The location of the user config.py file is now more portable across many different OSes and configuration styles, thanks to the use of the platformdirs helper package. There is also support for targets without the errno module, improved disconnect handling, and better ESP-device detection for USB-CDC ports.

The mpy_ld.py linker can now resolve fixed-address symbols if requested, for example on ESP8266 it can link to ROM-provided functions. There is also now support for ABS32 text relocations on ARMv6M (RP2040). These improvements extend the set of C extensions that can be compiled for such targets.

Libraries that have been updated in this release include: lwIP updated to STABLE-2_2_1_RELEASE, LittleFS updated to v2.11, libhydrogen updated to the latest release, and stm32lib updated to include support for STM32N6.

There have been many improvements to the test suite, to both the tests and the test runners. This is in part to aid the new Octoprobe test framework, which provides hardware-in-the-loop testing. The continuous integration testing now also includes an undefined behaviour sanitizer (UBSan) build, an address sanitizer (ASan) build, a long-long unix variant, and also testing of object representation C. MicroPython has a lot of tests!

The alif port now supports pin interrupts, has improved SPI transfers, and I2C configuration now allows changing the SCL and SDA pins.

The esp32 port is updated to use ESP-IDF v5.4.2, and now supports ESP32-C2. Most ESP32-based boards now auto-detect the size of their flash on boot and automatically create an appropriate vfs partition based on the size of the flash. This allows the same firmware image to work on boards with various flash sizes. This port has also added a new esp32.PCNT class along with machine.Counter and machine.Encoder, which can count input edges, including motor rotation. The PWM class has been improved, and its API now matches other ports. There is support for the LAN8670 PHY, and a new esp32.idf_task_info() function (useful with the new utop package in micropython-lib). The UART.sendbreak() method has been rewritten so that it doesn't reconfigure the UART during its execution.

The nrf port has had fixes and improvements to its UART REPL to make it more robust, and work with mpremote. It also now uses the correct iRAM address for native code execution on nRF52 MCUs. The nrf implementation of machine.enable_irq() and machine.disable_irq() has been reworked so it uses the code common with all other ports, and this is a breaking change (on nrf boards only) due to the signature change of enable_irq(). Previously the signature was enable_irq(), and now the signature matches other ports, and the docs, and is enable_irq(state), where state is the return value from disable_irq().

The rp2 port now enables compressed error messages by default, which reduces firmware size by about 3000 bytes. The pico-sdk alarm pool is now used (again, instead of custom soft-time code) for power-saving delays, and lightsleep has been improved. Open drain mode has been fixed on RP2350 with more than 32 GPIOs, and support for hard IRQ timer callbacks has been added.

The webassembly port has seen improvements to its FFI interface with JavaScript: it improves "has" and "get" proxying, fixes binding of self to JavaScript methods, implements equality for JsProxy objects, and reuses JsProxy references when possible to improve equality relationships of JavaScript objects on the Python side.

The zephyr port has been updated to use Zephyr v4.0.0, and many improvements have been made: PWM support has been added, UARTs are now interrupt driven with ring-buffers and can set the baudrate and other parameters, GPIO supports open-drain mode, SoftI2C and SoftSPI have been enabled, and the zephyr.FlashArea class now contains constants which enumerate the available partitions. REPL reliability has also been improved and it now supports ctrl-C in the default configuration. BLE can now create services at runtime using the standard BLE.register_services() method. Other small things have been added to make the zephyr port match the behaviour of other bare-metal ports, including: boot.py and main.py are now executed at start-up, /lib is added to sys.path as appropriate, stdin/out/err have been enabled in the sys module, along with the ability to import .mpy files, and await/async keywords are enabled. T...

Read more

ROMFS, alif port, RISCV inline assembler, DTLS, mpremote recursive remove

15 Apr 14:53
Compare
Choose a tag to compare

After more than three years in development, the "ROMFS" feature has been finalised, its filesystem format specified, and the VFS driver and supporting code are included in this release of MicroPython. This feature builds on bytecode version 6 (available for many years now), which supports executing bytecode in-place, that is, without the need to copy it to RAM. ROMFS defines a read-only, memory-mappable, extensible filesystem that can contain arbitrary resources, including precompiled mpy files, and allows executing bytecode directly from the filesystem. This makes importing significantly faster and use a lot less memory. Also, data resources such as fonts can be used in-place on a ROMFS without loading into RAM.

ROMFS is currently enabled only on selected boards: PYBD-SFx, all alif-port boards, a new ESP8266_GENERIC variant called FLASH_2M_ROMFS, and all stm32 Arduino boards. Other boards will have ROMFS enabled in the future, or it can be manually enabled on user-defined boards.

To build and deploy a ROMFS, mpremote has a new mpremote romfs command, with "query", "build", and "deploy" sub-commands, which can build and deploy a directory structure to a ROMFS partition on a target device. These initial ROMFS features will be extended in the future, but for now they provide a way to try out this long-anticipated feature.

This release also introduces a brand new "alif" port supporting Alif Ensemble MCUs. These MCUs offer multiple ARM cores, including Ethos-U55 machine-learning processors, and a comprehensive set of peripherals. Current features of the MicroPython alif port include USB support via TinyUSB, dual-core support using OpenAMP, octal SPI flash with XIP, the machine classes Pin, UART, SPI and I2C, and cyw43 WiFi and BLE support. Two alif board definitions are currently available: ALIF_ENSEMBLE for the official Alif Ensemble E7 DevKit, and OPENMV_AE3 for OpenMV's upcoming AE3-based camera board.

MicroPython's inline assembler now supports 32-bit RISC-V assembly code via the newly implemented @micropython.asm_rv32 decorator. This allows writing small snippets of RISC-V machine code that can be called directly from Python code. It is enabled on the rp2 port when the RP2350 is running in RISC-V mode.

Datagram TLS (DTLS) is now supported by the tls module and enabled on the alif, mimxrt, renesas-ra, rp2, stm32 and unix ports. An SSLContext can be created in DTLS mode using tls.PROTOCOL_DTLS_CLIENT or tls.PROTOCOL_DTLS_SERVER as the mode option, and this context can then be used to wrap a normal UDP socket to get a secure UDP connection.

The mpremote command-line tool now supports recursive remove via the new rm -r option; for example, mpremote rm -rv : can be used to remove all files and directories in the current working directory of the target device. mpremote also now supports relative URLs in the package.json file, installing from the local filesystem, and has optimised readline support in mpremote mount.

Improvements to the core interpreter include: full support for tuples and start/end arguments in the str.startswith() and str.endswith() methods; enabling of the two-argument version of the built-in next() function on most of the ports; a new sys.implementation._build entry which holds the build name of the target; and vfs.mount() with no arguments now returns a list of mounted filesystems.

The marshal module has been added with dumps() and loads() functions, which currently support code objects and, in combination with function.__code__, allow converting functions to/from a bytes object. This module is not enabled by default but can be used in custom build configurations.

The MicroPython native linker mpy_ld.py now includes support for linking in static libraries automatically. This allows the native-module build scripts to look for required symbols from libraries such as libgcc and libm that are provided by the compiler. This now makes it possible to use standard C functions like exp() in native modules. Also, native modules now support 32-bit RISC-V code.

The esp32 port now supports IDF v5.3 and v5.4, and support for versions below v5.2.0 has been dropped. Dynamic USB device support is now enabled on ESP32-S2 and ESP32-S3 MCUs, allowing configuration of the USB device at runtime. I2S has been enabled on all ESP32-C3 boards, the Pin.toggle() method has been added, and the I2C bus identifier is now an optional argument (by default, bus 0 is used). Additionally, memory management has been improved for the allocation of TLS sockets to attempt to automatically free any existing unused TLS memory when needed.

The mimxrt port now enables exFAT filesystem support and the PPP driver for boards with lwIP networking, and has support for a UF2 bootloader, making it easier to deploy firmware. The machine.RTC.now() method has been dropped (use datetime() instead), ADC.read_uv() has been added, and machine.I2C has support for the timeout keyword argument. The I2C, SPI and UART classes now support default buses, so the first argument to these constructors is no longer needed if the default bus is used. Some inconsistencies with PWM output have been fixed, along with an allocation bug for the UART RX and TX buffers.

The rp2 port sees the introduction of many new RP2350 boards, including the Pico 2 W, as well as support for PSRAM with size autodetection. The PIO interface now supports side_pindir selection, and SPI allows the MISO pin to be unspecified. Both the I2C and SPI classes now have the bus identifier as an optional argument with a default based on the board configuration. WPA3 is now supported on the Pico W and Pico 2 W in both AP and STA modes. Lost WiFi events due to code executing on the second core have now been fixed, mDNS has been fixed, and rp2.bootsel_button() and USB sleep now work on RP2350. ROMFS support has been added but is not enabled on any board by default; see commit 50a7362 for information on how to enable it manually.

The samd port has added full support for 9-bit data in the UART peripheral and supports default buses and pins for I2C, SPI and UART. DAC for two channels has been fixed on SAMD51, and UART buffering has had a few bug fixes.

The stm32 port now deinitialises I2C and SPI buses on soft-reset, which may be a breaking change for certain applications; be sure to always initialise I2C and SPI instances when creating them. The CAN code has been refactored, and a few minor bugs have been fixed there. Corrupt littlefs filesystems are now handled properly at startup: instead of a failed mount leading to a hard fault, the code attempts to mount again with default block device parameters, and if that also fails, it prints a message and continues the boot process. ROMFS is enabled on PYBD-SFx boards and all Arduino boards and can be enabled on other boards by manual configuration; see commit bea7645 for details. The PYBD-SF6 firmware now supports both original boards and new boards with larger SPI flash. WPA3 is now supported on boards using the cyw43-driver. mboot now includes a version string which is placed at the very end of the flash section allocated for this bootloader (usually 32k); this version can be retrieved using the fwupdate.get_mboot_version() function.

The zephyr port has had machine.Timer and machine.WDT implemented.

New boards added in this release are: ALIF_ENSEMBLE and OPENMV_AE3 (alif port), MAKERDIARY_RT1011_NANO_KIT (mimxrt port), MACHDYNE_WERKZEUG, RPI_PICO2_W, SEEED_XIAO_RP2350, SPARKFUN_IOTNODE_LORAWAN_RP2350, SPARKFUN_IOTREDBOARD_RP2350, SPARKFUN_PROMICRO_RP2350, SPARKFUN_THINGPLUS_RP2350, SPARKFUN_XRP_CONTROLLER, SPARKFUN_XRP_CONTROLLER_BETA and WEACTSTUDIO_RP2350B_CORE (rp2 port), ADAFRUIT_NEOKEY_TRINKEY, ADAFRUIT_QTPY_SAMD21, SAMD_GENERIC_D21X18, SAMD_GENERIC_D51X19 and SAMD_GENERIC_D51X2 (samd port), WEACT_F411_BLACKPILL (stm32 port).

The change in code size since the previous release for select builds of various ports is (absolute and percentage change in the text section):

   bare-arm:     +4  +0.007%
minimal x86:    -90  -0.049%
   unix x64: +16941  +2.046%
      stm32:    -96  -0.025%
     cc3200:   +280  +0.152%
    esp8266:   +964  +0.138%
      esp32: +10956  +0.654%
     mimxrt:  +7508  +2.065%
 renesas-ra:   -160  -0.026%
        nrf:   +168  +0.090%
        rp2:  +7944  +0.872%
       samd:  +1112  +0.418%

The leading causes of these changes in code size are:

  • minimal, stm32, renesas-ra: various small code-size optimisations
  • unix: enable VfsRom, update mbedTLS to v3.6.2, enable DTLS
  • cc3200: implement Pin.toggle() method
  • esp8266: enable function attributes, implement Pin.toggle(), allow enumerating connected stations in AP mode, update requests package
  • esp32: lots of small fixes and improvements
  • mimxrt: enable exFAT, add function.__code__ and function constructor
  • nrf: various small features such as sys.implementation._build, two-argument built-in next(), no-argument vfs.mount()
  • rp2: update mbedTLS to v3.6.2, enable DTLS, update cyw43-driver to 1.1.0
  • samd: support UART 9-bit data, add function.__code__ and function constructor, provide default IDs for UART, I2C and SPI

Performance of the VM and runtime is effectively unchanged since the previous release.

Thanks to everyone who contributed to this release: Alessandro Gatti, Alex Brudner, Amirreza Hamzavi, Andrew Leech, Angus Gratton, Anson Mansfield, Carl Pottle, Christian Clauss, chuangjinglu, Corran Webster, Damien George, danicampora, Daniël van de Giessen, Dryw Wade, eggfly, Garry W, garywill, Glenn Moloney, Glenn Strauss, Graeme Winter, Hans Maerki, Herwin Grobben, I. Tomita, iabdalkader, IhorNehrutsa, Jan Klusáček, Jan Sturm, Jared Hancock, Jeff Epler, Jon Nordby, Jos Verlinde,...

Read more

Patch release for mpremote, rp2 IGMP, esp32 PWM, SDCard, and AP channel

29 Nov 23:59
Compare
Choose a tag to compare

This is a patch release containing the following commits:

  • tools/mpremote: fix UnboundLocalError in Transport.fs_writefile()
  • esp32/machine_pwm: use IDF functions to calculate resolution correctly
  • pic16bit: make it build with recent XC16 versions
  • py/objdeque: fix buffer overflow in deque_subscr
  • extmod/modlwip: fix IGMP address type when IPv6 is enabled
  • esp32/machine_pwm: restore PWM support for ESP-IDF v5.0.x and v5.1.x
  • esp32: workaround native code execution crash on ESP32-S2
  • tools/mpremote: make sure stdout and stderr output appear in order
  • tools/mpremote: add test for forced copy
  • tools/mpremote: support trailing slash on dest for non-recursive copy
  • esp32/modsocket: fix getaddrinfo hints to set AI_CANONNAME
  • extmod/vfs_blockdev: support bool return from Python read/write blocks
  • extmod/network_cyw43: fix isconnected() result on AP interface
  • extmod/network_cyw43: fix uninitialised variable in status('stations')
  • extmod/network_cyw43: allow configuring active AP interface
  • esp32: fix setting WLAN channel in AP mode
  • esp32: use hardware version for touchpad macro defines
  • esp32: fix machine.TouchPad startup on ESP32-S2 and S3
  • extmod/modframebuf: fix 0 radius bug in FrameBuffer.ellipse
  • nrf/drivers/ticker: reset slow ticker callback count on soft reboot
  • py/objfloat: workaround non-constant NAN definition on Windows MSVC

RP2350 and ESP32-C6 support, RISC-V native emitter, common TinyUSB code

26 Oct 12:07
Compare
Choose a tag to compare

This release of MicroPython adds support for the new RP2350 MCU, improved RISC-V support with native code generation, support for ESP32-C6 MCUs, update Zephyr version with threading support, unified TinyUSB bindings across ports, a portable UART IRQ API, and enhanced mpremote recursive copy. There are also numerous bug fixes, enhancements to the test suite and more attention to testing of the machine module and its API. Read on for more details.

Support for the RISC-V architecture has been significantly extended, to include an RV32IMC native code emitter, native NLR and GC register scanning implementations for 32- and 64-bit RISC-V, support for placing RV32IMC native code in .mpy files and also freezing it, and RISC-V semihosting support. Testing for RISC-V is done with the qemu and unix ports, and the support is utilised in the esp32 and rp2 ports.

There is now support concatenation of adjacent f-strings, as well as raw f-strings. There is also a new micropython.RingIO class which provides a stream interface to an efficient, thread-safe, byte-oriented ring-buffer implementation.

All the ports that use TinyUSB have now been unified to use the same shared helper code for CDC serial. This includes: esp32 (S2 and S3), mimxrt, renesas-ra, rp2 and samd ports. With this has come the useful feature that the startup CDC serial data is buffered and then sent to the host upon connection. This means that (among other things) the REPL banner and initial prompt is now seen on first connection to a board.

Most ports now support registering Python callbacks for UART IRQs, and the semantics for the callback are made as consistent as possible across the ports. The possible IRQs are IRQ_RX, IRQ_RXIDLE, IRQ_TXIDLE and IRQ_BREAK, and the ports that have added support for these are esp32, mimxrt, nrf, renesas-ra, rp2 and samd (stm32 already had this feature).

In the networking sub-system, a new API for configuring IP addresses and related settings has been added: the global network.ipconfig() function, and a nic.ipconfig() method on individual network interfaces. This new API supports both IPv4 and IPv6, with much more control compared to the original nic.ifconfig() method; the latter is still available for backwards compatibility, but ipconfig() is now preferred moving forward.

A portable network.PPP implementation has been added, based on lwIP. This is not enabled by default but can be added to custom boards that use bare-metal lwIP.

The machine.SoftSPI class now supports least-significant-bit (LSB) mode, as well as the existing MSB mode.

The behaviour of sys.exit() and raise SystemExit (which are equivalent) has been changed: previously this would terminate the running script and drop to the REPL, but now it terminates the running script and triggers a soft reset of the device. This is more consistent with the unix port, which exits completely on a SystemExit exception. This change in behaviour is a breaking change and may impact certain applications; see commit 69c25ea for details.

The mpremote tool has a new hashing ability, eg mpremote sha256sum <file> and also has enhanced recursive copy which first checks the hash and only updates the destination file if the hash is different. This makes copying a large directory to a device significantly faster, and enables a "sync" workflow, where small changes can be made to a large application and then quickly deployed to the device via the recursive copy.

The esp32 port now supports ESP-IDF v5.2.2, and with this comes support for ESP32-C6 MCUs. The RISC-V native emitter has been enabled on C3 and C6 MCUs. There have also been some important bug fixes, for handling of native code loaded from .mpy files, fixes for stack corruption, and a fix to I2S, among other things. Also, applications that use many TCP sockets in quick succession should see an improvement because there is now a hard limit on the number of active TCP sockets.

The qemu port has been renamed from qemu-arm to simply qemu, and now supports both ARM and RISC-V architectures. It has also been reworked to provide a REPL and to run tests via a pty serial port, emulating how tests are run on bare-metal targets.

The rp2 port has updated pico-sdk to v2.0.0, which brings support to MicroPython for the new RP2350 MCU, in both ARM and RISC-V mode, and in both the 30- and 48-pin variants. IPv6 has been enabled by default on this port, and optional network.PPP made available. The USB now remains active during machine.lightsleep(), and there have been many bug fixes.

The stm32 port also sees optional network.PPP support through lwIP, and support for octospi on STM32H7 MCUs. A build option has been added to put IRQ, flash and UART code in RAM, and this feature is enabled on boards with a UART REPL, to allow filesystem access to work on these boards without losing UART characters (eg when using mpremote to copy files to a board). Preemptive keyboard interrupt via PendSV has been removed: this was buggy and no longer necessary; see ece950d for details. The ARDUINO_PORTENTA_H7 and ARDUINO_NICLA_VISION boards have added support for the NXP SE05x secure element, with integration in mbedTLS.

The webassembly port has better asyncio support, including the ability to do a top-level await of Task and Event objects. There has also been enhancements to the proxying of objects between JavaScript and Python.

The zephyr port has updated to use Zephyr v3.7.0, threading has been implemented through the _thread module, the REPL now operates in non-blocking mode so it can process events/callbacks, some more Python features including big-integers have been enabled, and machine objects (eg Pin, I2C) can now be constructed using device-tree node labels.

New boards added in this release are: ESP32_GENERIC_C6, M5STACK_ATOMS3_LITE, M5STACK_NANOC6, OLIMEX_ESP32_EVB, UM_FEATHERS3NEO, UM_OMGS3, UM_RGBTOUCH_MINI and UM_TINYC6 (esp32 port), RPI_PICO2 (rp2 port), ARDUINO_OPTA (stm32 port).

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

   bare-arm:   +116  +0.203%
minimal x86:   +185  +0.100%
   unix x64:  +8994  +1.098%
      stm32:  +1028  +0.263%
     cc3200:  +1152  +0.625%
    esp8266:  +2968  +0.426%
      esp32: -53617  -3.101%
     mimxrt:  +1864  +0.513%
 renesas-ra:  +1536  +0.245%
        nrf:  +1460  +0.781%
        rp2:  +3592  +1.068%
       samd:  +2244  +0.845%

The leading causes of these changes in code size are:

  • bare-arm, minimal: fix int.to_bytes() buffer size checks
  • unix: enable GCM and ECDHE-RSA in mbedTLS config
  • stm32: add new RingIO class
  • cc3200: add network.ipconfig and WLAN.ipconfig
  • esp8266: add network.ipconfig, WLAN.ipconfig and RingIO class
  • esp32: update ESP-IDF from v5.0.5 to v5.2.2
  • mimxrt, renesas-ra, nrf, rp2, samd: RingIO class and UART.irq support

Performance is effectively unchanged since the previous release on all ports, except the unix port which sees some reduction in GC times due to no more root-pointer scanning of executable memory.

Thanks to everyone who contributed to this release: Adrian Higgins, Alessandro Gatti, Alexandre Iooss, Amirreza Hamzavi, Andrea Milazzo, Andrew Leech, Angus Gratton, Ayush Singh, cajt, Christian Walther, Corran Webster, Damien George, Dan Halbert, danicampora, David Lechner, dmfaria, Dryw Wade, Elvis Pfützenreuter, Felix Dörre, George Hopkins, Glenn Moloney, iabdalkader, IhorNehrutsa, Jared Hancock, Jason Kridner, Jim Mussared, Jon Foster, Jos Verlinde, Junwha, Laurens Valk, Lennart, Leo Chung, Matt Trentini, Matthias Blankertz, Maureen Helm, Michael Sawyer, Michael Vornovitsky, nspsck, Owen, Paul Grayson, Peter Harper, Peter Züger, Phil Howard, Plaque FCC, Rick Sorensen, robert-hh, Seon Rozenblum, shiggy, stijn, Sylvain Zimmer, Takeo Takahashi, Terence Stenvold, tharuka, Tim Weber, timdechant, Volodymyr Shymanskyy, Yoctopuce, ZodiusInfuser.

MicroPython is a global Open Source project, and contributions were made from the following timezones: -0700, -0600, -0500, -0400, -0300, +0000, +0100, +0200, +0300, +0330, +0530, +0800, +0900, +1000, +1100.

The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Espressif, Anaconda, Arduino, LEGO Education, OpenMV and Planet Innovation.

What follows is a detailed list of changes, generated from the git commit history, and organised into sections.

Main components

py core:

  • nlrrv32: add RISC-V RV32I NLR implementation
  • lexer: support concatenation of adjacent f-strings
  • lexer: support raw f-strings
  • objdeque: fix deque type flags based on option settings
  • asmrv32: add RISC-V RV32IMC native code emitter
  • emitnative: emit better load/store sequences for RISC-V RV32IMC
  • emitnative: add more DEBUG_printf statements
  • emitndebug: add native debug emitter
  • emitnative: place thrown value in dedicated local variable
  • emitnative: fix native async with
  • misc: move mp_clz and mp_ctz intrinsics into misc.h
  • objint: fix int.to_bytes() buffer size checks
  • objarray: fix buffer overflow in case of memory allocation failure
  • asmrv32: make some code sequences smaller
  • objint: try to convert big-int back to small-int after binary op
  • asmrv32: do not use binary literals
  • objstr: add new mp_obj_new_str_from_cstr() helper function
  • remove 5 TODOs in emitbc, objrange and repl
  • lexer: add static assert that token enum values all fit in a byte
  • sequence: remove unused len argument from mp_seq_extract_slice
  • gc: remove commented-out functions
  • objtype: avoid crash on calling members of uninitialized native type
  • objtype: validate super() arguments
  • misc: fix msvc and C++ compatibility
  • runtime: fix sel...
Read more

Dynamic USB devices, revamped webassembly port, openamp, tls, vfs modules

31 May 05:03
Compare
Choose a tag to compare

This release of MicroPython adds support for dynamic USB devices defined in Python, adds new openamp, tls and vfs modules, completely revamps the webassembly port to add proxying between JavaScript and Python, and implements significant code size optimisations for frozen modules. There are also many other enhancements and bug fixes. Keep reading for a more detailed summary of the main changes.

After a lot of design, development and testing, MicroPython now has full support for user-defined USB devices. The interface is via a new machine.USBDevice class, and this allows the user to specify the USB descriptors and implement Python callbacks for USB endpoint transfers. With this, any USB device can be implemented in pure Python. While the machine.USBDevice interface is low-level and complete, there is a higher-level USB library in micropython-lib that allows easier implementation of devices, with examples for keyboard, mouse, MIDI and serial CDC. This feature is currently available on rp2 and samd ports, and other ports will follow in the future.

Support for the OpenAMP (asymmetric multiprocessing) protocol has been added through the new openamp module. This allows MicroPython to control other CPU cores in the system, to load and start processes and communicate with them through endpoints. This feature is currently available on the mimxrt and stm32 ports.

Two other new modules have been added: vfs and tls. The vfs module contains all VFS (virtual filesystem) related functions and classes, such as mount, umount and VfsFat. These were originally in the os module but having them there is not compatible with CPython, so they have been moved to their own dedicated module. They still exist in the os module for now but will eventually be removed from there, so it's recommended to start using the vfs module from now on. Similarly the new tls module is an evolution of the ssl module, whereby all the existing functionality in ssl has been moved to the tls module. This is done because MicroPython's SSL interface is becoming increasingly different to CPython's and moving this SSL/TLS functionality to a new tls module gives it room to grow and obtain new features that are useful for embedded applications. And compatibility with normal Python is still retained via a pure Python implementation of the ssl module. One new feature in the new tls module is the ability to register a certificate verification callback.

Other additions include more methods on the deque object so it is doubly-ended and supports iteration, and support for half-float 'e' format in struct.pack/struct.unpack. Dynamic native modules have had some additional runtime methods exposed, and the .mpy sub-version has been increased from v6.2 to v6.3 (native code in .mpy files will need to be recompiled, but bytecode does not and is still compatible).

There has also been significant code size optimisations for frozen modules. A new internal mp_proto_fun_t type has been defined which allows the common case of bytecode functions (as opposed to native code) to be stored in frozen code without any additional overhead of the mp_raw_code_t descriptor structure. All firmware builds using frozen modules will see a significant decrease in size. Code size has also improved further for very small targets by adding an option to remove the qstr hash bytes.

Internally in the source code the "STATIC" macro definition has been removed. Code should now just use "static" instead. If you have C/C++ code that uses "STATIC" then either replace it with "static", or provide your own #define to define "STATIC" as "static". See commit decf8e6 for details.

Mbedtls has been updated to version 3.5.1. And network interface constants, such as IF_STA and SEC_WPA2, have now been consolidated across ports so they all live at the WLAN class level, for example network.WLAN.SEC_WPA2 (existing constants at the network module level have been retained for backwards compatibility, but the new ones should be preferred from now on).

The esp32 port has seen some important bug fixes in the BLE component, to deinitialise without crashing, and increase the BLE task stack size. This port also uses the new I2S IDF driver, and supports IDF 5.0.5 and 5.2. There is support to enter the bootloader via machine.bootloader() and a new esp32.mcu_temperature(), for ESP32-C3/S2/S3 devices.

The rp2 port has added the new machine.USBDevice dynamic USB driver, and has had firmware performance optimisations applied to critical parts of the runtime and VM, giving about a 10% performance boost. There is now direct memory access to PIO and SPI FIFOs via proxy arrays, and bugs have been fixed with threads, lightsleep and UART IRQ latency.

The stm32 port has integrated support for the new openamp module, which is enabled on all Arduino boards. And firmware for Arduino boards now freeze in additional Arduino-specific library code. There have been fixes for internal flash writes on STM32H5 and STM32H7 MCUs (bank selection and flashing of the last word in a buffer), an important fix to a SPI DMA caching bug, an I2C clock enable fix for I2C4 on STM32F7 MCUs, and a fix for the FDCAN source clock frequency on STM32G4 MCUs. Mboot has added support for a new raw filesystem, to allow simpler and more robust firmware updates.

The webassembly port has seen a significant overhaul in its structure, and is now built as a JavaScript .mjs module that exports a user-friendly JavaScript-level API which is inspired by the API provided by Pyodide (which is a version of CPython that runs in the browser). This change is motivated by the need to use MicroPython as an engine within Pyscript, which is a platform for Python in the browser. New features in the webassembly port include proxying of objects between Python and JavaScript, a js module to interface with the JavaScript namespace, a jsffi module for miscellaneous proxy helpers, top-level async code, JavaScript driven asyncio support, automatic growing of the Python heap, integration of JavaScript and Python finalisation for global memory management, more time module functions, and support for build variants (following the unix port).

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

   bare-arm:   -220  -0.383%
minimal x86:   -341  -0.184%
   unix x64: +20168  +2.527%
      stm32:  -1692  -0.430%
     cc3200:   +256  +0.139%
    esp8266:  -8880  -1.260%
      esp32:  -3328  -0.194%
     mimxrt:   -408  -0.112%
 renesas-ra:   -464  -0.074%
        nrf:   -640  -0.341%
        rp2:  +5380  +1.626%
       samd:  +3224  +1.229%

The leading causes of these changes in code size are:

  • bare-arm, minimal: disabling qstr hashing
  • unix: updating mbedtls to version 3.5.1
  • stm32: optimising size of frozen modules
  • cc3200: addition of new vfs module, and bug fixes in the VM and array type
  • esp8266: disabling unused MICROPY_DEBUG_PRINTERS and optimising frozen modules
  • esp32, mimxrt, renesas-ra, nrf: optimising frozen modules
  • rp2: addition of machine.USBDevice, enabling -O2 optimisations
  • samd: addition of machine.USBDevice

Performance is effectively unchanged since the previous release on all ports, except the rp2 port which sees a performance improvement of roughly 10%.

Thanks to everyone who contributed to this release: Amirreza Hamzavi, Andrew Leech, Angus Gratton, Brian Pugh, Carlosgg, Christian Walther, Damien George, Daniël van de Giessen, darc, Dash Peters, David Lechner, Felix Dörre, iabdalkader, IhorNehrutsa, Iksas, J. Neuschäfer, Jared Hancock, Jim Lipsey, Jim Mussared, Jochen Sprickerhof, Joey232, Jos Verlinde, Kwabena W. Agyeman, Maarten van der Schrieck, Matt Trentini, Matthias Urlichs, Michiel W. Beijen, MikeTeachman, Nicko van Someren, Olivier Lenoir, Phil Howard, Rick Sorensen, robert-hh, Sebastian Romero, Simon Wood, Stanislav Ponomarev, stijn, Takeo Takahashi, Trent Piepho, Trent Warlaven, Vonasmic, YAMAMOTO Takashi, Yoctopuce.

MicroPython is a global Open Source project, and contributions were made from the following timezones: -0800, -0700, -0600, -0500, -0300, +0000, +0100, +0200, +0330, +0400, +0900, +1000, +1100.

The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Espressif, Anaconda, Arduino, LEGO Education, OpenMV and Planet Innovation.

What follows is a detailed list of all changes, generated from the git commit history, and organised into sections.

Main components

all:

  • fix "reuse" and "overridden" spelling mistakes
  • update bindings, ports and tests for mbedtls v3.5.1
  • use mp_obj_malloc_with_finaliser everywhere it's applicable
  • remove the "STATIC" macro and just use "static" instead
  • prune trailing whitespace
  • update extmod, ports, examples to build with new berkeley-db lib
  • add pre-commit hook for codespell
  • update copyright year range to include 2024
  • ISSUE_TEMPLATE: convert issue templates to forms

py core:

  • py.mk: remove extra build dir created for frozen_content
  • qstr: add support for MICROPY_QSTR_BYTES_IN_HASH=0
  • mpconfig: disable qstr hashing at minimum feature level
  • builtinimport: simplify calls to stat_path()
  • compile: fix potential Py-stack overflow in try-finally with return
  • emitglue: reorder and resize members of mp_raw_code_t
  • emitglue: provide a truncated mp_raw_code_t for non-asm code
  • emitglue: simplify mp_raw_code_t's kind and scope_flags members
  • emitglue: introduce mp_proto_fun_t as a more general mp_raw_code_t
  • emitglue: include fun_data_len in mp_raw_code_t only when saving
  • makeversionhdr.py: reinstate MICROPY_GIT_HASH in mpversion.h
  • obj: change sizeof to offsetof in mp_obj_malloc_var m...
Read more

Patch release for rp2 DMA, UART and BLE, esp32 BLE, renesas-ra I2C

20 Feb 12:36
Compare
Choose a tag to compare

This is a patch release containing the following commits:

  • py/compile: fix potential Py-stack overflow in try-finally with return
  • extmod/asyncio: support gather of tasks that finish early
  • extmod/modssl_mbedtls: fix cipher iteration in SSLContext.get_ciphers
  • extmod/btstack: reset pending_value_handle before calling write-done cb
  • extmod/btstack: reset pending_value_handle before calling read-done cb
  • esp32/mpnimbleport: release the GIL while doing NimBLE port deinit
  • esp32: increase NimBLE task stack size and overflow detection headroom
  • mimxrt/modmachine: fix deepsleep wakeup pin ifdef
  • renesas-ra/ra: fix SysTick clock source
  • renesas-ra/boards/ARDUINO_PORTENTA_C33: fix the RTC clock source
  • renesas-ra/ra/ra_i2c: fix 1 byte and 2 bytes read issue
  • rp2/rp2_dma: fix fetching 'write' buffers for writing not reading
  • rp2/machine_uart: fix event wait in uart.flush() and uart.read()
  • rp2: change machine.I2S and rp2.DMA to use shared DMA IRQ handlers

Patch release for rp2 atomic mutex

05 Jan 02:21
Compare
Choose a tag to compare

This is a patch release to fix a race condition and potential deadlock in the rp2 port's mp_thread_begin_atomic_section() function, when the second core is in use.

SSL support in asyncio, sorted qstr pools, common machine module bindings

27 Dec 12:58
Compare
Choose a tag to compare

This release of MicroPython introduces SSL/TLS support to asyncio, for both the client and server sides. The interface matches CPython: asyncio.open_connection() and asyncio.start_serve() now both accept an ssl argument to supply an SSLContext object. As part of this, new methods were added to SSLContext to load certificates, and certificate date/time validation was enabled on all ports that use mbedTLS.

Qstr pools are now sorted, which provides a significant performance boost for qstr_find_strn(), which is called a lot during parsing and loading of .mpy files, as well as interning of string objects, which happens in most string methods that return new strings. The static pool (part of the .mpy ABI) isn't currently sorted, but could be in the future.

There have been many internal changes to the machine module (and on some ports the os module) to factor the Python bindings to a common location, reduce code duplication and make the API more consistent across all the ports. And a new boardgen.py script has been added to factor pin generation and enable a more consistent machine.Pin across ports. For consistency, the following user-facing changes have been made:

  • cc3200 port: The machine module gains soft_reset(), mem8, mem16, mem32 and Signal; it loses POWER_ON (replaced by PWRON_RESET). disable_irq() now returns an (opaque) integer rather than a bool, and enable_irq(state) must be passed an argument which is the return value of disable_irq(), rather than a bool. In the os module, dupterm() has been converted to use the common implementation and has semantics the same as other ports, and uname() is removed to save space (sys.version and sys.implementation can be used instead).

  • esp32 port: In the machine module, lightsleep() and deepsleep() no longer take the sleep keyword argument, instead it's positional to match other ports. Also, passing 0 here will now do a 0ms sleep instead of acting like nothing was passed. And reset_cause() no longer accepts any arguments (before it would just ignore them).

  • esp8266 port: machine.idle() now returns None instead of the time elapsed. The machine.WDT() constructor now takes keyword arguments, and accepts the timeout argument but raises an exception if it's not the default value (this port doesn't support changing the timeout).

  • mimxrt port: machine.freq() now accepts an argument but raises NotImplementedError, and machine.lightsleep() has been added but also just raises NotImplementedError (this is to make these functions use an implementation common to the other ports).

  • nrf port: The machine module gains unique_id() (returns an empty bytes object), freq() (raises NotImplementedError) and Signal. UART.sendbreak() is removed, but this method previously did nothing. The os.dupterm() function has changed to match the semantics used by all other ports (except it's restricted to accept only machine.UART objects).

  • qemu-arm port: The machine module gains soft_reset() and idle().

  • samd port: The machine.deepsleep() function now resets after sleeping.

  • unix port: Gains machine.soft_reset().

  • zephyr port: The machine module gains soft_reset(), mem8, mem16, and mem32. The UART class gains the following methods: init() which supports setting timeout and timeout_char, deinit() which does nothing, flush() which raises OSError(EINVAL) because it's not implemented, and any() and txdone() which both raise NotImplementedError.

The teensy port has been removed in this release. This port was largely unmaintained, had limited features (the only hardware support was for GPIO and timer, and no machine module), and only supported a small number of Teensy boards.

A new preview versioning scheme has been introduced, whereby non-release builds are a preview of the next, upcoming release. This scheme is compatible with semver and should help to eliminate confusion matching documentation and firmware version numbers, among other things.

Black has been replaced with ruff format as the Python code formatter. This required a few small changes to Python code, and now allows linting and formatting with ruff.

Bound method instances now support comparison and hashing, matching CPython semantics. The .mpy sub-version has been updated from 6.1 to 6.2 due to a change in the native .mpy ABI. A new option MICROPY_PREVIEW_VERSION_2 has been added which provides a way to enable features and changes slated for MicroPython 2.x, by running make MICROPY_PREVIEW_VERSION_2=1. This is an alternative to having a 2.x development branch, and any feature or change that needs to be "hidden" until 2.x will use this flag.

LittleFS has been updated to v2.8.1. The associated MicroPython VfsLfs2 driver can read existing LFS2 filesystems, but any writes will update the filesystem to a newer LFS2 version that cannot be read by older drivers, so take this into account when updating, for example update mboot first.

The VFS sub-system has a new file ioctl to set the read-buffer size, which is used by mpremote to significantly increase performance of the "mpremote mount" feature. Manifest files now allow registering an external library path via add_library(name, path). sys.stdout.buffer.write() now returns the actual number of bytes written (although this is complicated when output goes to multiple destinations).

The esp32 port has been updated to use IDF version 5.0.4, and the initial GC heap size tuned so that, after doubling the heap size, WiFi can still be started and an SSL connection made. RMT.source_freq() is now a class method, socket connect timeout has been implemented, RTC user memory is now preserved over most reset causes, and hashlib.md5 enabled.

The mimxrt port has RTC alarm/wakeup functionality added, along with support for machine.deepsleep().

The rp2 port sees the introduction of a new rp2.DMA class for control over DMA transfers. It has switched to use the same math library as other ports to get more accurate floating point behaviour, and enabled os.dupterm_notify() for WebREPL use. The TinyUSB stack is now scheduled to run from the IRQ handler (instead of polled in the VM) which slightly improves performance of the VM and USB. The port also makes better use of event scheduling and WFE to be more efficient. It also has added support for external ADC channels (for example when using the ninaw10 driver).

The stm32 port has improved support for STM32H5xx MCUs, including Ethernet support, frequency scaling with HSI, sleep mode and SD card support. The NUCLEO_WL55 board now freezes in the LoRa driver, the I2S driver has improved accuracy of the clock frequency, and mboot now supports Microsoft WCID to set the USB driver.

New boards added in this release are: UM_TINYWATCHS3 (esp32 port), POLOLU_3PI_2040_ROBOT, POLOLU_ZUMO_2040_ROBOT and SIL_RP2040_SHIM (rp2 port), NUCLEO_H563ZI (stm32 port).

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

   bare-arm:   +216  +0.381%
minimal x86:   +624  +0.340%
   unix x64:  +8283  +1.050%
      stm32:  +1368  +0.350%
     cc3200:  +1184  +0.649%
    esp8266:   +800  +0.114%
      esp32: +35348  +2.100%
     mimxrt:  +2172  +0.602%
 renesas-ra:    +96  +0.015%
        nrf:  +1460  +0.785%
        rp2:  +6100  +1.880%
       samd:  +1476  +0.568%

The changes that dominate these numbers are:

  • bare-arm, minimal: comparing and hashing bound methods, sorted qstr pools
  • unix: updating LittleFS to 2.8.1, enabling certificate date/time validation, adding SSLContext certificate methods, asyncio SSL support
  • stm32: sorted qstrs, updating LittleFS to 2.8.1, I2S clock frequency improvements, asyncio SSL support
  • cc3200: sorted qstrs, more machine module functions, use of the common os.dupterm implementation
  • esp32: switching ESP-IDF from 5.0.2 to 5.0.4
  • esp8266: updating LittleFS to 2.8.1
  • mimxrt: adding RTC alarm/wakeup functionality, updating LittleFS to 2.8.1, asyncio SSL support
  • nrf: updating LittleFS to 2.8.1, enabling machine.Signal, asyncio SSL support
  • rp2: using locally-provided math library, adding new rp2.DMA class
  • samd: sorted qstrs, updating LittleFS to 2.8.1, asyncio SSL support

With the new sorted qstr pools, performance is significantly improved for qstr-heavy operations, between +50% and +200% improvement. Other areas have their performance unchanged since the last release.

Thanks to everyone who contributed to this release: Alessandro Gatti, Andrew Leech, Angus Gratton, Carlosgg, Christian Walther, Damien George, Daniël van de Giessen, Elias Wimmer, Glenn Moloney, iabdalkader, Ihor Nehrutsa, Jeff Epler, Jim Mussared, Kwabena W. Agyeman, Maarten van der Schrieck, Mark Blakeney, Mathieu Serandour, Matthias Urlichs, MikeTeachman, Ned Konz, Nicko van Someren, Pascal Brunot, Patrick Van Oosterwijck, Paul Grayson, Peter Züger, Rene Straub, robert-hh, Scott Zhao, Sebastian Romero, Seon Rozenblum, stijn, Thomas Ackermann, Thomas Wenrich, ThomHPL, Trent Piepho.

Contributions were made from the following timezones: -0800, -0700, -0600, -0500, +0000, +0100, +0200, +1000, +1100.

The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Planet Innovation, Espressif, Arduino, LEGO Education and OpenMV.

What follows is a detailed list of changes, generated from the git commit history, and organised into sections.

Main components

all:

  • switch to new preview build versioning scheme
  • replace "black" with "ruff format"
  • update Python formatting to ruff-format

py core:

  • vm: don't emit warning when using "raise ... from None"
  • builtinevex: handle invalid filenames for execfile
  • objbo...
Read more

U-module renaming, deflate module, IDF 5, board variants and Pico-W BLE

06 Oct 00:09
Compare
Choose a tag to compare

This release of MicroPython sees the renaming of built-in modules to remove the u-prefix, a new deflate module with optional compression support, the introduction of board variants, switching of the esp32 port to use IDF 5 together with improved heap management, support for BLE on RPi Pico W boards, and STM32H5xx support. The project is also now using codespell and ruff to improve code quality. New boards added in this release are: ARDUINO_NANO_ESP32 and UM_NANOS3 (esp32 port), ADAFRUIT_METRO_M7 (mimxrt port), ARDUINO_PORTENTA_C33 and VK_RA6M5 (renesas-ra port), ADAFRUIT_METRO_M4_EXPRESS (samd port), NUCLEO_L4A6ZG and STM32H573I_DK (stm32 port).

The renaming of built-in modules to remove the u-prefix -- for example utime becomes time, uasyncio becomes asyncio -- is done to improve compatibility with CPython and eliminate confusion about whether to import the u-version or the non-u-version of the name. Now, one should just always import the non-u-version and no longer think about u-naming. The only remaining module with a u-prefix is uctypes because it is not compatible with the CPython ctypes modules. The following modules are still available via their u-names for backwards compatibility: array, asyncio, binascii, bluetooth, collections, cryptolib, errno, hashlib, heapq, io, json, machine, os, platform, random, re, select, socket, ssl, struct, time, websocket. These modules (except for asyncio) are also extensible and can be overridden by a file with the same name, eg time.py. To force the import of a built-in, one must first clear (and subsequently restore) sys.path; the ability to write to the sys.path attribute has also been implemented in this release. For further information see the discussion at #9018.

Furthermore, importing itself has been tweaked and optimised, and importing of built-ins no longer touch the filesystem, which makes a typical built-in import take ~0.15ms rather than 3-5ms. For modules that fail to import, they are now removed from sys.modules, allowing the import to be attempted again. This required adding "NLR jump callbacks" to efficiently run some code if an exception is raised.

Additional improvements to the core runtime include support for conversion specifiers in f-strings (e.g. !r), speeding up of incremental GC cycles by tracking the last used block, addition of a new MICROPY_GC_SPLIT_HEAP_AUTO "auto grow heap" mode and support for pad bytes in struct format. The documentation, examples and test have also seen general improvements and additions.

For the extended modules, the zlib C module has been removed and replaced with a new MicroPython-specific deflate module and DeflateIO class that is optimised to provide efficient streaming compression and decompression. The zlib (and gzip) modules are now implemented in pure Python on top of the deflate module. The timeq module has been removed, it existed only for a previous version of (u)asyncio and is no longer used. In the ssl module, SSLContext has been added to be more compatible with CPython. The select module now supports using system/POSIX poll when possible and the unix port now uses this implementation instead of its own one. That means the unix port can now select/poll on custom Python objects that implement the appropriate ioctl. The socket module now supports SO_BROADCAST, and sys.std*.buffer objects now exist on unix. There is also a new esp-hosted network driver for external ESP32-based network coprocessors.

mpy-cross now allows reading source files from stdin and writing compiled code to stdout.

The esp8266 and esp32 ports add support for the Espressif ESP-NOW protocol, and the ability to set/get the power saving mode of the WLAN hardware. The esp8266 port adds board-variant support, combining all boards into a single ESP8266_GENERIC with FLASH_512K, FLASH_1M and OTA variants.

The esp32 port has now switched exclusively to ESP-IDF 5, and all existing components have been updated to work with this new IDF (except esp32.hall_sensor() which has been removed). The FSM ULP has been enabled for S2 and S3 chips, sockets now support SO_BINDTODEVICE, and board-named pins and the Pin.board dict have been implemented. The MicroPython heap has been reworked on this port to support the large variety of memory configurations: it now starts at 64kbytes and automatically grows as needed, with new segments allocated from the IDF heap. This means that boards with SPIRAM have much faster GC collection times if only a small amount of RAM is used, all available RAM can be used if needed, and the IDF has access to enough RAM for things like SSL sockets. For more information see #12141. The esp32 port has also had its generic boards consolidated and renamed to ESP32_GENERIC, ESP32_GENERIC_C3, ESP32_GENERIC_S2 and ESP32_GENERIC_S3, and some now have variants such as SPIRAM.

The mimxrt port has fixes and improvements to PWM and Pin.irq, as well as support for UART hardware flow control and UART.deinit. It has also seen integration of WiFi via the CYW43 driver, and Bluetooth via NimBLE. The mbedTLS bindings have enabled time validation of certificates.

The renesas-ra port has changed board names to match the product name, updated to use FSP v4.4.0, added support for: RA6M5 MCUs, machine PWM, DAC, RNG and SDCard classes, TinyUSB, Bluetooth via NimBLE, networking via lwIP, and mbedTLS.

Bluetooth support has also been added to the rp2 port for the Pico W board. And the RPi boards have been renamed from PICO to RPI_PICO, and PICO_W to RPI_PICO_W. Lightsleep has been fixed on this port so it works while WiFi is powered off, and time.time_ns() now has microsecond resolution.

The samd port sees the addition of SPI and QSPI flash drivers to support filesystems on external flash.

The stm32 port add support for STM32H5xx MCUs, basic support for the OCTOSPI peripheral, and USB support for STM32L1xx MCUs. New functions have been added to the stm module to support the sub-GHz radio on STM32WL55.

In micropython-lib, an extensive LoRa module has been added along with drivers for SX126x and SX127x chipsets, and the STM32WL55. This module supports both synchronous and asynchronous (asyncio) mode. Also, as part of the u-module renaming, urequests has been renamed to requests (but for backwards compatibility "import urequests" still works for now).

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

   bare-arm:    +192  +0.340%
minimal x86:    +310  +0.169%
   unix x64:   +4784  +0.610%
      stm32:    -524  -0.134%
     cc3200:    +280  +0.154%
    esp8266:   +8016  +1.151%
      esp32: +112133  +7.293%
     mimxrt:   +3624  +1.015%
 renesas-ra:   -2184  -0.348%
        nrf:    +616  +0.332%
        rp2:   +1920  +0.595%
       samd:   -7904  -2.953%

The changes that dominate these numbers are:

  • bare-arm, minimal, cc3200, nrf: NLR jump callbacks and support for extensible modules
  • unix: update of mbedTLS to v2.28.1, support for polling Python objects in select module
  • stm32: removal of the timeq module
  • esp8266: addition of the espnow module
  • esp32: switching to ESP-IDF 5
  • mimxrt: UART features, time validation of SSL certificates
  • renesas-ra: move to FSP v4.4.0, remove timeq module
  • rp2: machine.PWM enhancements
  • samd: drop support for VfsLfs1

Performance is effectively unchanged since the previous release.

Note that this is the last release to use the current versioning scheme for nightly/unstable builds, whereby a build between releases is versioned as v1.20.0--g (following the release of v1.20.0). Moving forward, nightly builds will now be called preview builds and be versioned with the next release number. For example, if the last release was v1.21.0 then preview releases will be of the form v1.22.0-preview..g. For discussion see #12127.

Thanks to everyone who contributed to this release: Adam Green, Alexander Wilde, algonell, Andrew Leech, Andy Piper, Angus Gratton, Armin Brauns, brave ulysses, Brett Cannon, Brian 'redbeard' Harrington, Carlosgg, Chris Wilson, Christian Clauss, Damien George, Damien Tournoud, Daniël van de Giessen, David Grayson, David Lechner, David Yang, dotnfc, Duncan Lowther, Elecia White, elibdev, Elvis Pfutzenreuter, Felix Dörre, Francis Dela Cruz, Glenn Moloney, glenn20, iabdalkader, Ihor Nehrutsa, Jared Hancock, Jim Lipsey, Jim Mussared, Jon Nordby, Jonas Scharpf, Jos Verlinde, Kwabena W. Agyeman, Luca Burelli, marble, Mark Grosen, mbedNoobNinja, mcskatkat, Mingjie Shen, Mirko Vogt, Nicholas H. Tollervey, Oliver Joos, Ondrej Wisniewski, patrick, Peter Harper, Phil Howard, Philipp Ebensberger, Rene Straub, robert-hh, Sebastian Romero, Seon Rozenblum, stephanelsmith, stijn, Takeo Takahashi, Thomas, Tobias Thyrrestrup, UnexpectedMaker, Victor Rajewski, vsfos, Wang Xuancong, Wanlin Wang, Wilko Nienhaus, Wind-stormger, Yaroslav Halchenko, Yilin Sun, Yuuki NAGAO.

The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Planet Innovation, Espressif, Arduino, LEGO Education and OpenMV.

What follows is a detailed list of changes, generated from the git commit history, and organised into sections.

Main components

all:

  • fix spelling mistakes based on codespell check
  • fix strings with backslash by using raw string literals
  • fix various Python coding inconsistencies found by ruff
  • fix cases of Python variable assigned but never used
  • rename MP_QSTR_umodule to MP_QSTR_module everywhere
  • rename mp_umodule*, mp_module_umodule* to remove the "u" prefix
  • rename mod_umodule*, ^umodule* to remove the "u" prefix
  • rename UMODULE to MODULE in preprocessor/M...
Read more

New mip package manager, compressed type structs and Pico W support

26 Apr 15:20
Compare
Choose a tag to compare

This release of MicroPython introduces a new lightweight package manager called mip, which uses a custom protocol to query and install packages that is optimised for embedded systems. It is intended to take over the role of upip and supports installing packages from micropython-lib as well as any URL. Mip can be run directly on a device (with network connectivity) or via mpremote. As part of this, all pure-Python drivers have been moved from the micropython repository to the micropython-lib repository, making it easier to install the packages needed for a given project.

In the MicroPython runtime, core/built-in types have been compressed by only including in the C-level type struct as many slots for C function pointers as is needed for a given type (instead of storing NULL pointers for unused slots). This resulted in a reduction of code size of many kilobytes for all ports (eg around 1200 for bare-arm) with a very minimal loss in performance. Any third-party C extensions will need to be updated to work with this change. See commits 662b976 and 3ac8b58 for details.

The MicroPython VM/runtime has also seen many improvements and bug fixes, including: addition of hex/fromhex methods to bytes/memoryview/bytearray, dictionary union (PEP 584), support for __float__ and __complex__ functions, support for __dict__ on module objects and UTF-8 validation of all strings. Whole number floats are now formatted exactly, and there have been other accuracy improvements to parsing and formatting of floats. A sub-version has been introduced for .mpy files which is used only when a .mpy file includes native code, allowing the native ABI to change while retaining backwards compatibility with pure bytecode .mpy files. There is now an option to have the GC heap split across more than one pool of memory, although this is not yet in use by the ports. And root pointer registration has been simplified with the new MP_REGISTER_ROOT_POINTER macro.

In the network module, the keyword arguments to scan(), connect() and config() have been renamed to be more consistent across ports and drivers. The changed argument names are: "essid" changed to "ssid", "auth" or "authmode" changed to "security", and "password" changed to "key". The old keyword arguments are still supported so this change is backwards compatible. The functions network.hostname() and network.country() have also been added and provide a standard interface across ports to set the global networking configuration for all interfaces and interface types. In particular, setting the hostname is now supported for all CYW43xx-based boards (pyboard-D, Arduino Portenta, Pico W, and others).

The mbedtls (SSL library) settings have also been unified across ports that use this library and enable elliptic curve DH and DSA cryptography enabled so that devices can connect to more websites. The cert_reqs and cadata arguments have also been implemented. Sockets now support leaving multicast groups. WebREPL will now run the page from the device, allowing WebREPL resources to load from a HTTPS site.

The bluetooth module has a minor breaking change for IRQ_GATTC_CHARACTERISTIC_RESULT: def_handle has been replaced with end_handle. This is not expected to affect most programs as def_handle is unlikely to be used. See commit cacc96d.

The uasyncio module now supports stream read(-1), handles gather with no awaitables, and adds a clear() method to ThreadSafeFlag. The framebuf module adds ellipse and polygon drawing methods, and a fill argument to rect() for consistency with those new methods.

A new port has been added, the "embed" port, which is a port of MicroPython that targets the C language and outputs a set of self-contained .c and .h files for embedding into a wider project. An example is provided to show how this works.

The esp8266 port has reverse-special-methods enabled on the GENERIC board, uart.flush() and uart.txdone() added, real open drain output enabled on pins driven by PWM, and adds the ability to set TX power for the WLAN interface.

The esp32 port now uses synchronous BLE events which allows support for BLE pairing and bonding. The LAN driver adds support for LAN8710, KSZ8081, configuration of ETH ref_clk pin, and support for SPI-based Ethernet chips. UART now supports setting timeout_char and implements uart.flush() and uart.txdone(). New boards include ESP32S3 with octal SPIRAM, Olimex PoE boards and a generic board for unicore chips. There have also been many bug fixes, in particular for newer MCU versions such as S2, S3 and C3.

The mimxrt port has seen a lot of clean up of the code, support for MIMXRT1176 MCUs and the MIMXRT1170_EVK board, a soft-timer implementation of machine.Timer, addition of machine.bootloader(), as well as uart.flush() and uart.txdone().

On the nrf port, machine.UART and machine.PWM have been improved to match other ports (the PWM change is a breaking change), and machine.I2C now supports the freq argument. New boards include the Arduino Nano 33 BLE sense board and the Seeed XIAO NRF52840 Sense.

The rp2 port sees the integration of the CYW43xx WiFi driver which is used in the new Pico W board, a board similar to the Pico but with WiFi support. Other new boards include W5500_EVB_PICO, WEACTSTUDIO and the nullbits Bit-C PRO board. Support has been added for named pins, including Pin.board and Pin.cpu attributes, consistent with other ports that support this feature. The machine.lightsleep() function has been implemented, along with SSL certificate time validation, and more accurate formulas are used to determine PWM frequency and duty. The GC heap available to MicroPython programs has been increased to use the maximum available free RAM, adding about 20k to the user heap. There have also been important bug fixes to threading and concurrency when using both cores.

The samd port has been significantly extended to include: named pins with a pins.csv file, and Pin.board and Pin.cpu attributes; ADC, PWM, DAC, SoftSPI, SoftI2C, SPI, I2C and RTC classes added to the machine module; pin.irq() method and OPEN_DRAIN mode for pins; freq, disable_irq, enable_irq, idle, lightsleep and time_pulse_us functions added to the machine module. Floating point support has been added for SAMD21 devices, and the math module enabled for SAMD51. A board definition for Sparkfun Think Plus has been added.

The stm32 port sees support added for STM32L1xx MCUs, STM32H723, USB on STM32G0xx and mboot support for STM32G0xx. The existing CYW43xx WiFi driver has been replaced with the open-source version of cyw43-driver and this has been integrated with all existing boards with CYW43xx chips. Support for the CC3000 WiFi driver has been removed, and the Wiznet driver integration has been reworked to use the generic one shared with other ports (and also uses lib/wiznet5k instead of the old drivers/wiznet5k). Hardware I2C implementation has been added for STM32L1xx and STM32L4xx, support added for Arduino 1200bps touch to enter the bootloader (only on Arduino boards), mbedtls (SSL) certificate time validation added, and link-time-optimisation (LTO) enabled by default on boards with small flash size. New board definitions include: LEGO Hub No. 7, NUCLEO-L152RE, NUCLEO-F756ZG, NUCLEO-H723ZG, Arduino Nicla Vision and Arduino Giga H7. Arduino boards also now include a range of frozen libraries including senml.

For the unix and windows ports, build outputs (including executables) are now kept inside the $(BUILD) directory associated with the build. As part of this the variant suffix has been removed from the executable name. For example, what was micropython-coverage is now build-coverage/micropython, and the standard variant is build-standard/micropython. Furthermore, the dev variant has been removed and its features enabled instead on the standard variant, which now enables the same feature set as a typical bare-metal board, making it more convenient to use for development. The remaining variants are: minimal, standard, nanbox, coverage. The SSL implementation for this port has switched from axtls to mbedtls, again to match most bare-metal ports.

The javascript port has been renamed to the webassembly port, and adds support for VFS and VfsPosix using Emscripten's POSIX filesystem layer. The way stdout printing works has also changed to dispatch via a custom event called "micropython-print".

The zephyr port upgraded Zephyr to v3.1.0, added custom configuration for bbc_microbit_v2, and enabled finalisers.

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

   bare-arm:   -1124  -1.934%
minimal x86:   -4322  -2.967%
   unix x64: +264181 +50.533% standard
      stm32:   -3592  -0.906% PYBV10
     cc3200:   -1832  -0.990%
    esp8266:   -2284  -0.327% GENERIC
      esp32:   +9659  +0.632% GENERIC
        nrf:     -84  -0.045% pca10040
        rp2:  +13096  +4.291% PICO
       samd: +127776 +90.488% ADAFRUIT_ITSYBITSY_M4_EXPRESS

The changes that dominate these numbers are:

  • bare-arm, minimal, stm32, cc3200, esp8266: reduced size due to compressed type structs
  • unix: switching axtls for mbedtls, merging in dev configuration options and features
  • esp32: new Ethernet drivers and some additional machine module functionality
  • nrf: compressed type struct and machine module improvements
  • rp2: additional mbedtls features and ciphers
  • samd: many new features, and machine module classes and functions

Thanks to everyone who contributed to this release: Alex Riesen, Andrew Leech, Andrew Scheller, Angus Gratton, Antonello Margottini, Antonin ENFRUN, Ayke van Laethem, Blake Felt, brave ulysses, Brian Cooke, Brian Pugh, Carlosgg, Chris Overgaauw, Chris Swan, Chris Waggoner, chrisma...

Read more