uasyncio on esp8266 #141
uasyncio on esp8266 #141
Conversation
279487e to
28acc93
Compare
|
Thanks for the contribution. It's good to see that the only modifications to uasyncio is to do with the fd number. But I'm not sure if this particular PR will be merged because @pfalcon is working on it and possibly has an alternative solution. Anyway, we can keep this PR open until uasyncio is supported on esp8266. |
|
Yes, this is definitely step in the direction it should be done, but it should be done in right way, e.g. final implementation won't have any sock_fd() function. But before going to networking support on esp8266 (which will require many more changes than this patch), simpler things are being worked on. For example, the first stage is making sure uasyncio.core trivial scheduling can work w/o memory allocation (done), next stage is making the same work with uasyncio (simple as it seems, but subclassing breaks no-alloc behavior; in progress). Next stage isn't even related to uasyncio, but due to do deficiencies in HW API, all current samples for the previously described functionality are currently broken (don't work as expected from a board to board), so that needs to be fixed. Then can patch unix port uselect module to accept entire objects, and the more complex task of adding efficient buffering for partial reads so uasyncio networking can actually work as expected. @mzdaniel , you're welcome to test changes to uasyncio* being landed. Btw, I'd be curious how performance was affected when you ran tests (https://github.com/micropython/micropython-lib/tree/master/uasyncio/benchmark) against this patch? |
|
Great to meet you, Damien and Paul. @dpgeorge: Sure. As a python instructor, uasyncio enables async/await semantics which is wonderful.
Certainly. It is done this way as stopgap to facilitate testing in esp8266.
Not sure how reliable is this data, as was taken from an xubuntu in GUI mode, each one freshly rebooted. Tried quick single user mode, and to my surprise python event loop doesn't seem to be available? (happen in both micropython and asyncio) I've also found unittest is in the lib and preparing a PR there with updates to help us testing. |
28acc93 to
ea26971
Compare
|
PR updated now that micropython/micropython#1550 (comment) and specifically micropython/micropython@093a8f5 has landed. It includes a simple integration unittest to help us with testing. The test runs under linux, though not yet on esp8266. Here is the latest run of boom-heavy using ea26971 and Tests on esp8266 with PResp8266 is not super stable but it works. ab barely completed 50 single Interestingly, if instead test_http_server_light.py is run as a paste to the StandaloneWith terminal |
| @@ -63,9 +64,9 @@ def wait(self, delay): | |||
| # Remove "if res" workaround after | |||
| # https://github.com/micropython/micropython/issues/2716 fixed. | |||
| if res: | |||
| for fd, ev in res: | |||
| cb = self.objmap[fd] | |||
| if DEBUG and __debug__: | |||
There was a problem hiding this comment.
When fixing conflict, you dropped upstream change.
There was a problem hiding this comment.
Thank you for catching it! Didn't see it in the diffs somehow.
|
Happy New Year! @mzdaniel : Thanks for running the tests and further updates. While this would an interim solution, I guess it's good idea to merge this patch after few more minor changes. Besides inline comments, please squash together "Add minimum esp8266 support" and "Improve socket handling using changes from unix/moduselect" (but leave the test a separate commit). Regarding the test, thanks for adding it, there's definitely a need for more tests (I have few pseudo-tests locally, not suitable for committing.) One issue though is that micropython-lib follows pattern where tests are located within a module's dir, not in tests/ subdir. So, unless you have a good argument for what would be an inconsistency, please move it up. Thanks! |
|
I may need more time to respond to your test results in detail, but as a quick hint, benchmark (load) tests are intended to be run on unix, esp8266 simply doesn't have enough memory to sustain so many connections (there's a chance to address that (somewhat) by violating TCP/IP protocol, and that's definitely on TODO, but of course not a priority ;-) ). |
ea26971 to
4bc5ab9
Compare
|
Happy New Year too, Paul!
My pleasure. It's done.
IMHO, aside of what seems a convention (to put tests in its own directory) in other important python projects that feels cleaner to me (in the sense of visual overload in the root dir of the module) and more convenient for test automation, I found the following: instead if it is in tests: I introduced test_echo.py here as I felt is important and if agreed, decide details and what to do. Of course, local conventions and your preference have priority, so it is your call. |
Right, indeed, in my local copy I have a symlink to make that work.
Yes, but only if you installed modules first. If you made changes in micropython-lib, but didn't install them, you may be testing old versions. So, this solution has drawbacks too. Anyway, as there're some "tests" (which aren't really tests, but that's another issue) in base dir already (https://github.com/micropython/micropython-lib/tree/master/uasyncio), and you've made this change in the patch, let it be in the base dir for now. And I perfectly know about tests/ convention, and that works well for more or less big, standalone projects. But micropython-lib is different, it's myriad of small modules, and usually it's just a module .py. Then having test_*.py side by side with it allows to test always-fresh module without bothering with python path. Of course, it doesn't work as convinient and as clean with packages consisting of multiple submodules, like uasyncio or umqtt. |
|
Merged, thanks! (I don't retest thoroughly now, assuming you did that.) |
|
Thank you all! |
Thank you so much Damien, Paul and the MicroPython community for this amazing project!
Since I read about this project and found a few weeks back that python3.5 can be run in the esp8266 (with less than 64kb of memory), I was immediatelly hook. Once esp8266 gained uselect it seemed that with a little more effort uasyncio was in reach and made a POC (https://github.com/mzdaniel/micropython-lib/tree/uasyncio-esp8266_poc). Here is the revised PR after utimeq landed on the esp. Tested with the latest MicroPython v1.8.6-247-g05aebb9-dirty on 2016-12-26 on an Olimex board.