ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF, Renesas-RA and SAMD ports. #14041

robert-hh · 2024-03-06T07:27:07Z

Adds UART IRQ functionality for 4 additional ports: RP2, ESP32, MIMXRT and SAMD. Available triggers:

IRQ_RX: called every time when a byte is received. Not useful for high data rates (like > 9600 baud).
IRQ_RXIDLE: called when the receive line gets silent after a burst of data. Very useful!
IRQ_TXIDLE: called when a block of data has been sent.

Not all ports provide all triggers:

rp2: IRQ_RXIDLE, IRQ_TXIDLE and IRQ_BREAK. IRQ_RXIDLE works as it should. IRQ_TXIDLE is only called when the data length is mor than 5 bytes, and it triggers when still 4-5 bytes are to be sent.
MIMXRT: IRQ_RXIDLE and IRQ_TXIDLE. Work as they should with a fix to the NXP UART driver. which is included in this PR.
SAMD: IRQ_RX and IRQ_TXIDLE. The SAMD port's hardware does not support IRQ_RXIDLE. Based on the IRQ_RX trigger a simple class can be made which implements a IRQ_RXIDLE handler call. IRQ_TXIDLE triggers while the last byte is sent.
ESP32: IRQ_RX and IRQ_BREAK: The ESP32 does not support IRQ_RXIDLE. The option hard=True is not supported.
NRF: IRQ_RX and IRQ_TXIDLE. Enabled for boards with nrf52480. I could not test other boards.

Tested with RP2040, ESP32. ESP32-S2, ESP32-S3, ESP32-C3, SAMD21, SAMD51, NRF52840, and all supported MIMXRT MCUs except i.MX RT 1064 (I do not have access to a board).

Test observations for other boards:

STM32: IRQ_RXIDLE. For F4xx MCU (e.g. Pyboard), the RXIDLE interrupt triggers twice with every message longer than 1 byte starting with the second message after UART init. The first trigger happens at the first byte of the message, the second after the message. 1 Byte messages get only one trigger, but with different delays after the message. Tests with F7xx and H7xx boards showed proper behavior.
Renesas-RA: Trigger IRQ_RX.
CC2300: Trigger RX_ANY. IRQ_RX was added as alias in this PR.

dpgeorge · 2024-03-06T08:21:05Z

Thanks for this, it's a great addition! Definitely needed.

By chance I was working on the docs side of this today: #14040.

robert-hh · 2024-03-11T11:48:35Z

@dpgeorge I extended the documentation by adding the commit from your PR #14040.

robert-hh · 2024-03-11T17:29:25Z

Discussion topics:

Some alternative names for the triggers:
IRQ_RXANY instead of IRQ_RX
IRQ_TXDONE instead of IRQ_TXIDLE
The trigger IRQ_TXIDLE for the nrf port seems of less value, since uart.write() is synchronous and waits anyhow, until all data has been sent. That's because the nrf implementation does not use separate buffers for writing. But that synchronous mode can be changed easily, such that uart.write() returns immediately. The drawback then is, that the data which was sent must not be changed until the write is finished, which can be enforced with uart.flush(), tested with uart.txdone() or released by a IRQ_TXIDLE callback.

github-actions · 2024-05-24T19:49:05Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +8 +0.002% PYBV10
     mimxrt:  +408 +0.112% TEENSY40
        rp2:  +632 +0.070% RPI_PICO_W[incl +8(bss)]
       samd:  +736 +0.276% ADAFRUIT_ITSYBITSY_M4_EXPRESS

dpgeorge · 2024-06-21T07:38:32Z

Thanks again for working on this. After thinking more about it here are my general comments.

I don't think it makes sense to have both IRQ_RX and IRQ_RXIDLE, especially because it seems that the ports either implement one or the other, but never both.

We need to make it as easy as possible to write portable code that runs across all the ports, and having different IRQs, or the same IRQ but with slightly different semantics, makes that hard.

So I suggest to not have IRQ_RX at all, and make IRQ_RXIDLE mandatory. Users should be able to rely on thisIRQ to have the following semantics:

IRQ_RXIDLE: every character received by the UART will have this interrupt
triggered at some point in time after that character is received

The idea is that if you only ever do uart.read() within the IRQ_RXIDLE callback, then you should always be able to (eventually) read all available characters. And ideally in a timely manner.

Even if the hardware/SDK doesn't support this concept we should still be able to synthesize it. Eg if the hardware supports IRQ_RX then in combination with a timer we can implement IRQ_RXIDLE.

Having IRQ_BREAK (when the hardware supports it) makes sense, the semantics for that seem obvious.

For IRQ_TXIDLE/IRQ_TXDONE: as with IRQ_RXIDLE the semantics of this need to be well specified, and then all ports should implement it.

robert-hh · 2024-06-21T09:21:25Z

So you suggest that IRQ_RX is renamed to IRQ_RXIDLE for those ports that do not support a true IRQ_RXIDLE, and for those that do so IRQ_RX is dropped?

B.t.w: I made two Python classes as example that implement IRQ_RXIDLE based on an existing IRQ_RX. The simpler one is below with uart.irq() being a method. The other implements uart.irq() as a class. I considered implementing that in the machine_uart module itself, but dealing with the timer is easier in Python.

#
# Simple wrapper class for UART, adding the trigger IRQ_RXIDLE for port
# which just support IRQ_RX
# Missing: Handling of the irq.xxx methods in this class.
#
import machine

class UART(machine.UART):
    def __init__(self, id, *args, **kwargs):
        self.irq_timeout_count = 0
        self.irq_rx_count = 0
        self.timeout_idle = 5
        self.irq_timer = None
        self.irq_handler = None
        self.IRQ_RXIDLE = 4096

    def irq(self, handler=None, trigger=None, hard=False, period=5):
        if handler is not None:
            if trigger == self.IRQ_RXIDLE:
                self.irq_handler = handler
                self.irq_timer = machine.Timer(callback=self.irq_timer_callback,
                    mode=machine.Timer.PERIODIC, period=period)
                return super().irq(handler=self.irq_rx_callback, trigger=self.IRQ_RX, hard=True)
            else:
                if self.irq_timer is not None:
                    self.irq_timer.deinit()
                self.irq_handler = None
                return super().irq(handler=handler, trigger=trigger, hard=hard)
        else:
            if self.irq_timer is not None:
                self.irq_timer.deinit()
            self.irq_handler = None
            return super().irq(handler=None, trigger=0, hard=False)

    def irq_rx_callback(self, uart):
        self.irq_rx_count += 1
        self.irq_timeout_count = 0

    def irq_timer_callback(self, timer):
        if self.irq_rx_count > 0:
            self.irq_timeout_count += 1
            if self.irq_timeout_count == 2:
                if self.irq_handler is not None:
                    self.irq_handler(self)
                self.irq_rx_count = 0
                self.irq_timeout_count = 0

robert-hh · 2024-06-23T18:40:53Z

@dpgeorge Thinking about further, I prefer to support the behaviour with proper names. Looking at the list of ports, three of them support true IRQ_RXIDLE, and eight support IRQ_RX. Both the rp2040 and the MIMXRT port can support IRQ_RX mode. The rp2040 commit would get much simpler and more close to the existing state. Without support for IRQ_RXIDLE, the MIMXRT port would not need a change to the NXP UART driver.
So I suggest to have IRQ_RX as the common method and have support for IRQ_RXIDLE as a derived class method.

robert-hh · 2024-08-28T10:31:54Z

The change to esp32/machine_timer.c is made and the MIMXRT MCU's re-tested. I removed at the test scripts the restrictions for Teensy 4.x. So any MIMXRT board can be used.

Supported trigger names: IRQ_RXIDLE, IRQ_TXIDLE, IRQ_BREAK - IRQ_RXIDLE: The handler for IRQ_RXIDLE is called reliably 31 UART bit times after the last incoming data. - IRQ_TXIDLE: This IRQ is triggered after at least >5 characters are sent at once. It is triggered when the TX FIFO falls below 4 elements. At that time, up to 5 bytes may still be in the FIFO and output shift register. - IRQ_BREAK: The IRQ triggers if a BREAK state is detected at RX. Properties & side effects: - After a BREAK, a valid character must be received before another break can be detected. - Each break puts a 0xff character into the input buffer. The irq.flags() value is cleared only with a new wanted event. Do not change the flags otherwise. Signed-off-by: robert-hh <robert@hammelrath.com>

Supported for all SAMD51 devices and SAMD21 with external flash. For interrupt events, IRQ_RX and IRQ_TXIDLE are provided. IRQ_RX is called for every received byte. This may not be useful for high data rates, but can be used to build a wrapper class providing an IRQ_RXIDLE event or to signal just the first byte of a message. IRQ_TXIDLE is called only when messages are longer than 5 bytes and triggers when still 5 bytes are due to be sent. The SAMD hardware does not support implementing IRQ_RXIDLE. Signed-off-by: robert-hh <robert@hammelrath.com>

Supported triggers are: IRQ_RXIDLE and IRQ_TXIDLE. When IRQ_RXIDLE is set, the handler will be called 3 character times after the data in burst stopped. When IRQ_TXIDLE is set, the handler will be called immediately after the data has been sent. This commit requires a change to fsl_lpuart.c, because the existing code does not support under-run appropriately. The irq.flags() value is cleared only at an expected event. Do not change it otherwise. Signed-off-by: robert-hh <robert@hammelrath.com>

The renesas-ra port supports calling a handler to be called on every byte received by UART. For consistency with other ports, the symbol IRQ_RX is added as the trigger name. Side change: Add the received UART data to the REPL input buffer only if it is the REPL UART. Otherwise, every UART would act as REPL input. Signed-off-by: robert-hh <robert@hammelrath.com>

As alternative to RX_ANY to match the names used by the other ports. Signed-off-by: robert-hh <robert@hammelrath.com>

Supported trigger events: IRQ_RX and IRQ_BREAK. Hard IRQ is not supported. Signed-off-by: robert-hh <robert@hammelrath.com>

This commit fixes a bug in the existing driver, that the UART baud rate could not be changed without reset or power cycle. It adds as well functionality to UART.deinit(). Signed-off-by: robert-hh <robert@hammelrath.com>

Supported triggers: UART.IRQ_RX and UART.IRQ_TXIDLE. It will probably work on other boards as well, but so far untested. The irq.flags() value is changed only when requested by a triggered event. Do not change it otherwise. Signed-off-by: robert-hh <robert@hammelrath.com>

Just adding the event symbol. No code change required, and no impact on code execution time when the event is not selected. Tested with STM32F4xx, STM32F7xx and STM32H7xx. Signed-off-by: robert-hh <robert@hammelrath.com>

With the softtimer the minimal delay between the end of a message and the trigger is 2 ms. For baud rates <= 9600 baud it's three character times. Tested with baud rates up tp 115200 baud. The timer used for RXIDLE is running only during UART receive, saving execution cycles when the timer is not needed. The irq.flags() value is changed only with an expected event. Do not change it otherwise. Signed-off-by: robert-hh <robert@hammelrath.com>

Allowing to define the trigger UART.IRQ_RXIDLE as well as UART.IRQ_RX. The delay for the IRQ_RXIDLE interrupt is about 3 character times or 1-2 ms, whichever is larger. The irq.flags() value is changed only with an expected event. Do not change it otherwise. Signed-off-by: robert-hh <robert@hammelrath.com>

The UART.IRQ_IDLE callback is called about two character times after the last byte, or 1 ms, whichever is larger. For the irq, timer 0 is used. machine_timer.c had to be reworked to make it's mechanisms available for machine_uart.c. The irq.flags() value is change only at a requested event. Otherwise keep the state. Signed-off-by: robert-hh <robert@hammelrath.com>

These docs now match the code in `extmod/machine_uart.c`. IRQ trigger support still need to be updated for each port (to be done in a follow-up commit). Signed-off-by: Damien George <damien@micropython.org>

For more ports and trigger options, based on the current state of the code. Signed-off-by: robert-hh <robert@hammelrath.com>

The test checks whether the message created by the IRQ handler appears about at the end of the data sent by UART. Supported MCUs resp. boards: - RP2040 - Teensy 4.x - Adafruit ItsyBitsy M0 - Adafruit ItsyBitsy M4 - NRF52 (Arduino Nano Connect 33 BLE) Signed-off-by: Damien George <damien@micropython.org>

These all require hardware connections, so live in a different directory. Except for the IRQ_BREAK test of ESP32 devices a single UART with loopback is sufficient. General: SAMD21: Due to the limited flash size only SAMD21 devices with external flash support uart.irq(). IRQ_BREAK: ESP32 needs different UART devices for creating and sensing a break. Lacking a second UART the test is skipped for ESP32S2 and ESP32C3. RP2 does not pass the test reliable at 115200 baud, reason to be found. Thus the upper limit is set to 57600 Baud. Coverage: esp32 pass when different UART devices are used. rp2 pass up to 57600 baud IRQ_RX: SAMD21: Being a slow device it needs data to be sent byte-by-byte at 9600 baud, since the IRQ callback is scheduled delayed and then the flags do not match any more. The data matches since it is queued in the FIFO resp. ringbuffer. CC3200: The test cannot be performed since no calls are accepted in the IRQ handler like u.read(). Skipped. Coverage: cc3200 fail due to major differences in the implementation. esp32 pass nrf pass renesas-ra pass samd pass see the notes. stm32 pass IRQ_RXIDLE: STM32: With PyBoard the IRQ is called several times, but only once with the flag IRQ_RXIDLE set. Coverage: esp32 pass mimxrt pass renesas-ra pass rp2 pass samd pass for both SAMD21 and SAMD51 stm32 fail. see notes. Signed-off-by: Damien George <damien@micropython.org> Signed-off-by: robert-hh <robert@hammelrath.com>

dpgeorge · 2024-08-29T07:00:42Z

I removed at the test scripts the restrictions for Teensy 4.x. So any MIMXRT board can be used.

OK, great. I also removed the test configuration for the samd SEEED_XIAO_SAMD21 because that doesn't have UART IRQs enabled by default (I was enabling them manually to test on this boad). I have now ordered an ADAFRUIT_ITSYBITSY_M0_EXPRESS to use for testing instead.

I also did some very minor fixes to this PR:

remove some stray double blank lines
add a few blank lines around #if/#endif's
make const mp_irq_methods_t uart_irq_methods static

And I kept all 16 commits. They are well separated commits making independent changes.

Thanks @robert-hh for all your work on this! And @Gadgetoid for testing, and @projectgus for additional reviews.

robert-hh · 2024-08-29T07:27:42Z

Thank you for the improvements and for merging.

iabdalkader · 2024-10-27T10:06:01Z

Note this doesn't build anymore with more recent SDKs because the isr variable is now defined:

extern lpuart_isr_t s_lpuartIsr[];

robert-hh · 2024-10-27T10:13:15Z

Note this doesn't build anymore with more recent SDKs

Which port?

iabdalkader · 2024-10-27T10:14:37Z

Which port?

mimxrt, with any recent SDK release. They changed that variable to any array.

robert-hh · 2024-10-27T10:18:25Z

We are still at sdk 2.10, and we use a modified fsl_lpuart.c to get a proper IRQ_RXIDLE behavior. Updating to a newer SDK is still due. When that happens I expect more changes to be done. No too many, as I noticed when trying that last year.

iabdalkader · 2024-10-27T10:32:10Z

We are still at sdk 2.10, and we use a modified fsl_lpuart.c to get a proper IRQ_RXIDLE behavior.

I understand. I'm just saying if you try to use a more recent SDK this doesn't build anymore. However, it seems everything else works. I'm not sure which SDK I'm using but it's more recent than the one in MicroPython.

dpgeorge added port-samd port-mimxrt port-rp2 labels Mar 6, 2024

robert-hh changed the title ~~posrt: Support UART.irq for the RP2, MIMXRT and SAMD port.~~ ports: Support UART.irq for the RP2, MIMXRT and SAMD port. Mar 6, 2024

robert-hh force-pushed the ports_uart_irq branch 6 times, most recently from 1c2854a to 27742b0 Compare March 10, 2024 14:54

robert-hh changed the title ~~ports: Support UART.irq for the RP2, MIMXRT and SAMD port.~~ ports: Support UART.irq for the RP2, ESP32, MIMXRT and SAMD port. Mar 10, 2024

robert-hh added the port-esp32 label Mar 10, 2024

robert-hh force-pushed the ports_uart_irq branch from 27742b0 to b18b64b Compare March 11, 2024 07:14

robert-hh force-pushed the ports_uart_irq branch from ce8fc89 to 7d14aee Compare March 11, 2024 15:50

robert-hh added the port-nrf label Mar 11, 2024

robert-hh changed the title ~~ports: Support UART.irq for the RP2, ESP32, MIMXRT and SAMD port.~~ ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF and SAMD ports. Mar 11, 2024

robert-hh force-pushed the ports_uart_irq branch from 7d14aee to 2754f50 Compare March 12, 2024 10:53

robert-hh mentioned this pull request Apr 16, 2024

docs: machine.UART.irq arguments are incorrect #14311

Closed

2 tasks

robert-hh mentioned this pull request May 10, 2024

extmod: Add generic network.PPP to work with lwIP #14461

Merged

robert-hh force-pushed the ports_uart_irq branch from 2754f50 to f28671f Compare May 24, 2024 19:43

robert-hh force-pushed the ports_uart_irq branch 3 times, most recently from 9c59a7c to 0a0831b Compare June 1, 2024 04:48

robert-hh and others added 16 commits August 29, 2024 16:27

cc3200/mods/pybuart: Add the UART.IRQ_RX class constant.

1027b5f

As alternative to RX_ANY to match the names used by the other ports. Signed-off-by: robert-hh <robert@hammelrath.com>

esp32/machine_uart: Implement Python UART IRQ with IRQ_RX and IRQ_BREAK.

a04a141

Supported trigger events: IRQ_RX and IRQ_BREAK. Hard IRQ is not supported. Signed-off-by: robert-hh <robert@hammelrath.com>

stm32/machine_uart: Add the UART.IRQ_RX event for UART.irq().

a86619f

Just adding the event symbol. No code change required, and no impact on code execution time when the event is not selected. Tested with STM32F4xx, STM32F7xx and STM32H7xx. Signed-off-by: robert-hh <robert@hammelrath.com>

docs/library/machine.UART: Fix UART.irq docs to match current code.

9bbe616

These docs now match the code in `extmod/machine_uart.c`. IRQ trigger support still need to be updated for each port (to be done in a follow-up commit). Signed-off-by: Damien George <damien@micropython.org>

docs/library/machine.UART: Extend the documentation for UART.irq.

03b1b6d

For more ports and trigger options, based on the current state of the code. Signed-off-by: robert-hh <robert@hammelrath.com>

dpgeorge force-pushed the ports_uart_irq branch from dc934f7 to 09d070a Compare August 29, 2024 06:51

dpgeorge merged commit 09d070a into micropython:master Aug 29, 2024
40 checks passed

robert-hh deleted the ports_uart_irq branch August 29, 2024 07:43

projectgus mentioned this pull request Sep 26, 2024

rp2: Fix dropped UART REPL bytes on soft reset. #15914

Merged

robert-hh mentioned this pull request Nov 3, 2024

ports: mimxrt: Switch to official GitHub SDK (WIP) #11516

Draft

DvdGiessen mentioned this pull request Apr 16, 2025

esp32/machine_uart: Correctly manage UART queue and event task. #17135

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF, Renesas-RA and SAMD ports. #14041

ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF, Renesas-RA and SAMD ports. #14041

robert-hh commented Mar 6, 2024 •

edited

Loading

dpgeorge commented Mar 6, 2024

robert-hh commented Mar 11, 2024

robert-hh commented Mar 11, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

dpgeorge commented Jun 21, 2024

robert-hh commented Jun 21, 2024

robert-hh commented Jun 23, 2024

robert-hh commented Aug 28, 2024

dpgeorge commented Aug 29, 2024

robert-hh commented Aug 29, 2024

iabdalkader commented Oct 27, 2024

robert-hh commented Oct 27, 2024

iabdalkader commented Oct 27, 2024 •

edited

Loading

robert-hh commented Oct 27, 2024

iabdalkader commented Oct 27, 2024

ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF, Renesas-RA and SAMD ports. #14041

ports: Support UART.irq for the RP2, ESP32, MIMXRT, NRF, Renesas-RA and SAMD ports. #14041

Conversation

robert-hh commented Mar 6, 2024 • edited Loading

dpgeorge commented Mar 6, 2024

robert-hh commented Mar 11, 2024

robert-hh commented Mar 11, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

dpgeorge commented Jun 21, 2024

robert-hh commented Jun 21, 2024

robert-hh commented Jun 23, 2024

robert-hh commented Aug 28, 2024

dpgeorge commented Aug 29, 2024

robert-hh commented Aug 29, 2024

iabdalkader commented Oct 27, 2024

robert-hh commented Oct 27, 2024

iabdalkader commented Oct 27, 2024 • edited Loading

robert-hh commented Oct 27, 2024

iabdalkader commented Oct 27, 2024

robert-hh commented Mar 6, 2024 •

edited

Loading

robert-hh commented Mar 11, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

iabdalkader commented Oct 27, 2024 •

edited

Loading