extmod/uasyncio: Add ThreadSafeEvent. #6886

jimmo · 2021-02-12T03:45:13Z

This is a MicroPython-extension that allows for events to be set from IRQ (hard or soft) or scheduler context.

Works on the bare-metal ports (which use extmod/uselect). To use this on Unix requires PR #6885

dpgeorge · 2021-02-13T04:27:30Z

This new class is self-clearing, which is different to how Event behaves (the latter has a clear() method). IMO this class should match Event. But then there's also the question of whether this ThreadSafeEvent should allow multiple tasks to wait on it (like Event). That of course adds code, and is not strictly needed (one could use a standard Event combined with this one to multiplex). If it's not going to match Event then maybe rename it to something like ThreadSafeFlag?

dpgeorge · 2021-02-13T04:28:48Z

extmod/uasyncio/event.py

+
+# MicroPython-extension: Event class that can be set from IRQs or scheduler context.
+# Implementation is a stream that asyncio will poll until a flag is set.
+class ThreadSafeEvent(io.IOBase):


hmm... this now puts a requirement on all ports to have uio.IOBase... is it worth guarding this code in hasattr(uio, 'IOBase')?

dpgeorge · 2021-02-13T04:29:21Z

extmod/uasyncio/event.py

@@ -29,3 +33,24 @@ async def wait(self):
            core.cur_task.data = self.waiting
            yield
        return True
+
+
+_MP_STREAM_POLL = const(3)


to eliminate the need for a from micropython import const I'd say just remove this line and hard-code the value with a comment below

dpgeorge · 2021-02-13T04:29:40Z

extmod/uasyncio/event.py

+        self._flag = 0
+
+    def ioctl(self, req, flags):
+        if req == _MP_STREAM_POLL:


this could be hard-coded as: if req == 3: # MP_STREAM_POLL

jimmo · 2021-02-15T06:32:39Z

OK that all sounds very reasonable -- I've update it to ThreadSafeFlag and added docs and a basic test. I've kept it self-clearing because it's hard to avoid races otherwise.

The test is a bit annoying because it can't run on Unix (see #6885), but I've run it on pyboard and on a unix build with extmod/uselect.

jimmo · 2021-02-16T04:40:26Z

Updated to use import uio instead of import io.

dpgeorge · 2021-02-16T04:58:37Z

extmod/uasyncio/event.py

+        async def wait(self):
+            yield core._io_queue.queue_read(self)
+            self._flag = 0
+except ImportError:


2x blank lines needed here for black formatting

dpgeorge · 2021-02-16T05:33:21Z

docs/library/uasyncio.rst

+.. method:: ThreadSafeFlag.wait()
+
+    Wait for the flag to be set.  If the flag is already set then it returns
+    immediately.


that doesn't match the implementation (returning immediately if set)... maybe the implementation should be changed so it does return immediately if self._flag is set... that would make things more responsive at the risk of "starving" other things (but since Event.wait already behaves this way we know it's probably ok to return immediately)

Yeah I agree. I've updated the code to return immediately and added that to the test.

This is a MicroPython-extension that allows for code running in IRQ (hard or soft) or scheduler context to sequence asyncio code. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

dpgeorge · 2021-02-16T06:01:20Z

tests/extmod/uasyncio_threadsafeflag.py.exp

+yield
+task 3
+wait task
+task 3 done


when I run this test on a PYBD-SF6 I get:

set event yield task 3 task 3 done wait task

which makes more sense to me than this .exp file, that task 3 starts and is immediately finished

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

dpgeorge · 2021-02-16T06:22:09Z

Thanks!

peterhinch · 2021-02-16T13:44:08Z

Very nice. I'll do some testing later.

dpgeorge · 2021-02-16T14:15:35Z

You can make an async-pin like this:

class AsyncPin:
    def __init__(self, pin, trigger):
        self.pin = pin
        self.flag = ThreadSafeFlag()
        self.pin.irq(lambda pin: self.flag.set(), trigger, hard=True)

    def wait_edge(self):
        return self.flag.wait()

Then call await async_pin.wait_edge().

tve · 2021-02-16T18:37:03Z

How does this fit with uevent plans?

dpgeorge · 2021-02-17T00:49:06Z

How does this fit with uevent plans?

I still have uevent on the to-do list, it's just been blocked by cmake and other things (which are being cleared now). I hope that we can still offer the same ThreadSafeFlag class and API after the move to uevent, and it'll become efficient (rather than need polling).

Fix espressif run modes

dpgeorge added the extmod Relates to extmod/ directory in source label Feb 13, 2021

dpgeorge reviewed Feb 13, 2021

View reviewed changes

jimmo force-pushed the asyncio-polling-event branch from 60d5f7c to 965cbff Compare February 15, 2021 06:29

jimmo force-pushed the asyncio-polling-event branch 2 times, most recently from 5f4bd8b to e2eefd4 Compare February 16, 2021 04:39

dpgeorge reviewed Feb 16, 2021

View reviewed changes

jimmo force-pushed the asyncio-polling-event branch from e2eefd4 to 6d5f4c6 Compare February 16, 2021 04:59

dpgeorge reviewed Feb 16, 2021

View reviewed changes

jimmo added 2 commits February 16, 2021 16:35

extmod/uasyncio: Add ThreadSafeFlag.

5e96e89

This is a MicroPython-extension that allows for code running in IRQ (hard or soft) or scheduler context to sequence asyncio code. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

docs/library/uasyncio.rst: Add docs for ThreadSafeFlag.

cdf9c86

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo force-pushed the asyncio-polling-event branch from 6d5f4c6 to 51e0a39 Compare February 16, 2021 05:41

dpgeorge reviewed Feb 16, 2021

View reviewed changes

tests/extmod: Add test for ThreadSafeFlag.

83d2305

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo force-pushed the asyncio-polling-event branch from 51e0a39 to 83d2305 Compare February 16, 2021 06:08

dpgeorge merged commit 83d2305 into micropython:master Feb 16, 2021

scy mentioned this pull request Mar 16, 2021

extmod/uasyncio Interface to ISR is problematic #6415

Closed

Wind-stormger pushed a commit to BPI-STEAM/micropython that referenced this pull request Sep 15, 2022

Merge pull request micropython#6886 from MicroDev1/update-run-mode

33a1006

Fix espressif run modes

bogdanm mentioned this pull request Jun 22, 2023

Zephyr port: async event loop implementation starves CPU #11837

Open

Uh oh!

extmod/uasyncio: Add ThreadSafeEvent. #6886

extmod/uasyncio: Add ThreadSafeEvent. #6886

Uh oh!

Conversation

jimmo commented Feb 12, 2021

Uh oh!

dpgeorge commented Feb 13, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jimmo commented Feb 15, 2021

Uh oh!

jimmo commented Feb 16, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpgeorge commented Feb 16, 2021

Uh oh!

peterhinch commented Feb 16, 2021

Uh oh!

dpgeorge commented Feb 16, 2021

Uh oh!

tve commented Feb 16, 2021

Uh oh!

dpgeorge commented Feb 17, 2021

Uh oh!

Uh oh!