Skip to content

ESP32 RMT Implementation #5184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 20 commits into from

Conversation

mattytrentini
Copy link
Contributor

@mattytrentini mattytrentini commented Oct 5, 2019

This is the start of a MicroPython implementation for RMT on the ESP32.

RMT allows accurate (down to 12.5ns) sending and receiving of pulses. This implementation currently only provides transmit features and there are some limitations:

  • Blocking only (send_pulses is now non-blocking)
    • Non-blocking is possible but increases complexity
    • May need some help looking at how to implement the ISR and pass that back as an optional callback...
  • Looping is not exposed (it's possible to loop a pattern indefinitely)
    -Requires non-blocking
  • Carrier features - selecting a base frequency that can be modulated - is not yet provided
  • Idle level is not configurable
  • Only one memory block per channel

(Update: non-blocking and looping have been implemented)

I'd be curious which of these features is most important to people.

The short-term intent is to allow this to be used to provide a solid NeoPixel implementation. On the ESP32 it's difficult to make toggle pins accurately enough since the operating system periodically interrupts. The RMT provides accurate timing independent of the OS.

Note that FastLED, a popular C library for controlling Neopixels, successfully uses RMT for the ESP32 implementation.

Basic usage:

>>> import esp32
>>> from machine import Pin
>>> r = esp32.RMT(1, pin=Pin(18), clock_div=80)
>>> r
RMT(channel=1, pin=18, clock_div=80)
>>> r.send_pulses((100, 2000, 100, 4000), start=0)  # Send 0 for 100*1e-06s, 1 for 2000*1e-06s etc

The length of the buffer is restricted only by available memory.

Managing resources is not-quite-right yet (once an RMT channel has been allocated it isn't deintialised on soft boot) but I expect to address that soon.

From PyCom's RMT documentation (take care not to inspect their implementation since it's an incompatible license) they appear to offer three different ways to configure a tx buffer. The second model is similar to the one implemented here. Which, if any, of the others would be of most use?

For reference there are a few related issues:
#5117
#5157
#3453

Although this RMT implementation isn't complete yet I think it is in a useful state and I thought it worthwhile to submit and gather feedback for the future direction.

Comment on lines 371 to 372
The RMT is a module, specific to the ESP32, that allows pulses of very accurate interval
and duration to be sent and received::
Copy link
Contributor

@mcauser mcauser Oct 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expand this paragraph to show RMT is short for Remote Control and that it was originally built for IR but can be repurposed.

The RMT (Remote Control) module, specific to the ESP32, was designed to send and receive infrared remote control signals. Due to flexibility of module, the driver can also be used to generate or receive many other types of digital signals. The module allows pulses of very accurate interval and duration to be sent and received::

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion! I've updated the paragraph accordingly.

@mcauser
Copy link
Contributor

mcauser commented Oct 5, 2019

Using this driver, I am able to generate a signal for a FS1000A ASK/OOK 433MHz transmitter to ring an Aldi Delta doorbell.

# convert binary data to pulses
def build_pulses(data, long, short, reset, preamble):
    pulses = [preamble, short, reset]
    one = [long, short]
    zero = [short, long]
    for c in reversed(data):
        pulses[1:1] = one if c == '1' else zero
    return pulses

# convert int data to zero padded binary string
def data_to_str(data, len):
    s = ('{:0%db}' % len).format(data)
    return s[-len:]

# transmit
def tx(data, len, repeats):
    if not isinstance(data, str):
        data = data_to_str(data, len)
    pulses = build_pulses(data, 1070, 358, 11070, 100)
    for _ in range(repeats):
        r.send_pulses(pulses)

# main
import esp32
from machine import Pin
r = esp32.RMT(1, Pin(18), 80)

# data = "100111000001110000000001"
data = 10230785
data_len = 24 # bits
repeats = 20

tx(data, data_len, repeats)
# doorbell rings!!
TinyPICO FS1000A
18 Data
3V3 VCC
GND GND

fs1000a

urh

@MrSurly
Copy link
Contributor

MrSurly commented Oct 7, 2019

Although this RMT implementation isn't complete yet I think it is in a useful state and I thought it worthwhile to submit and gather feedback for the future direction.

I'm wondering if this could be used to parse RC PPM (multiple servo channels on one wire), and then drive servos?

I have an old robot that I built that receives all the inputs from the RC TX via PPM, decodes them, then has to drive the wheels using normal RC PWM signals. There's little direct coupling between them because there's a decent amount of math that converts the incoming signals to the proper motor drive. The original runs on a teensy, and I've since lost the code. Would be neat to re-implement in MP.

@UnexpectedMaker
Copy link
Contributor

UnexpectedMaker commented Oct 9, 2019

I got this working driving panels and strips of GRBW and GRB ws2812's in my live stream this morning and it's working great! We had some teething problems yesterday, but once @mattytrentini worked out how to force the idle state to be low, everything fell into place.

We definitely need a way of having deinit be auto called on an initialised channel when code fails, or at least have it clear itself on a soft reboot. Right now if the code fails before deinit is called, a hard reset is required to clear it again.

Or even just a:
deinit( channel )
So deinit can be called to clear a channel before an init attempt.

Also, the default error that appears when it's not cleared and you try to re-init is vague:
OSError: ESP_ERR_INVALID_STATE

Otherwise, I'm excited to keep using this and start building a new modern RGB LED driver library for the ESP32!

@mattytrentini
Copy link
Contributor Author

I got this working driving panels and strips of GRBW and GRB ws2812's in my live stream this morning and it's working great! We had some teething problems yesterday, but once @mattytrentini worked out how to force the idle state to be low, everything fell into place.

I have pushed that (one character!) fix. Thanks for finding it! A better fix will be to expose the idle level to allow it to be configured for the channel. Will get on to that.

(BTW it's really not clear to me what the purpose is of setting config.tx_config.idle_output_en to 0 is...)

We definitely need a way of having deinit be auto called on an initialised channel when code fails, or at least have it clear itself on a soft reboot. Right now if the code fails before deinit is called, a hard reset is required to clear it again.

I hear you, this is the next issue on my list. :)

Or even just a:
deinit( channel )
So deinit can be called to clear a channel before an init attempt.

Oh, there is a deinit: r.deinit(). Should probably document that 😛

Otherwise, I'm excited to keep using this and start building a new modern RGB LED driver library for the ESP32!

Thanks for sticking with it when there were issues!

@nevercast
Copy link
Contributor

nevercast commented Oct 9, 2019

BTW it's really not clear to me what the purpose is of setting config.tx_config.idle_output_en to 0 is...

That manipulates register RMT_IDLE_OUT_EN_CHn, which I gather disables the output enable driver on the GPIO so that the pin is not driven. This is specultation.

You could test the idle_en feature by disabling it and see if you can sink or source any current through that pin.

I noticed the build is failing with a code size issue, @mcauser was having a similar issue with 4 bytes being the problem, the solution was to rebase on to master.

#5118 (comment)

@mattytrentini
Copy link
Contributor Author

I'm wondering if this could be used to parse RC PPM (multiple servo channels on one wire), and then drive servos?

Obviously we'd need to add receive capabilities but yes, from that link you sent, I believe RMT could decode PPM.

I guess that means receive functionality would be high on your list of requests? ;)

@UnexpectedMaker
Copy link
Contributor

UnexpectedMaker commented Oct 9, 2019

Oh, there is a deinit: r.deinit(). Should probably document that 😛

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

I'm after a deinit ( channel number ) - to clear a channel, without having a reference to the channel object.

Is that even a Pythonic thing to do? ;)

@nevercast
Copy link
Contributor

Oh, there is a deinit: r.deinit(). Should probably document that

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

Maybe its worth not throwing an error when we init, just reinit. I can call Pin.init as many times as I like. Can we think of anywhere else in MicroPython where calling .init or constructing a machine object throws instead of gracefully reinit ?

@UnexpectedMaker
Copy link
Contributor

Oh, there is a deinit: r.deinit(). Should probably document that

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

Maybe its worth not throwing an error when we init, just reinit. I can call Pin.init as many times as I like. Can we think of anywhere else in MicroPython where calling .init or constructing a machine object throws instead of gracefully reinit ?

That sounds even better! Can haz plz?

@nevercast
Copy link
Contributor

Umm. Did you merge instead or rebase? That's a lotta commits

@mattytrentini
Copy link
Contributor Author

Apologies, in my rush to rebase yesterday I’ve messed something up. Will try to sort it out asap.

@mattytrentini mattytrentini force-pushed the esp32_rmt branch 2 times, most recently from ea53ec6 to 630f491 Compare October 11, 2019 00:46
@nevercast
Copy link
Contributor

image

@mattytrentini
Copy link
Contributor Author

Apologies, in my rush to rebase yesterday I’ve messed something up. Will try to sort it out asap.

Should be sorted out now, huge thanks to @mcauser for help fixing the screwed-up rebase!

@nevercast
Copy link
Contributor

Hi @mattytrentini

Do you want to develop this incrementally and get the TX functionality landed in master earlier, or do you want to continue developing your future features first and then bring it all in at once?

@nevercast
Copy link
Contributor

Have tried this RMT code with my own OOK transmitter that just arrived from China, and its a bundle of joy!
image

@mattytrentini
Copy link
Contributor Author

@nevercast I'd prefer to get it in early. Correctly managing resources is critical (and, until fixed, must prevent this being merged) but other transmit methods - different ways to specify the pulse pattern as well as specifying using the carrier methods - are nice-to-have.

Receive can come later; I'm guessing that API will be more difficult to get right.

@dpgeorge
Copy link
Member

Also, I'm still worried about the freq/divider discussion. At the end, the resolution is important. So why not specifying that?

So there are 3 options that I can think of:

  1. Stay close to what the IDF exposes / the hardware, and use source_freq and clock_div.
  2. Provide a minor level of abstraction via resolution_ps (picoseconds!).
  3. Follow how machine.Timer works and use tick_hz and period, such that the resolution is period/tick_hz. Could default tick_hz=1_000_000_000 then period would default to nanoseconds.

Options 2 and 3 require using very large integers. The arguments for and against 1 vs 2 are discussed in comments above.

Would be good to agree on what to do here...

@mattytrentini
Copy link
Contributor Author

I'm biased ;) but IMO option 1 is the only one without problematic implementation issues.

If we want to provide higher-level abstractions we can do it above the RMT interface; for example, provide a function to return a clock divider for a given desired resolution. Perhaps return a tuple of (source_freq, clock_div, actual_resolution) for a given desired_resolution?

@dpgeorge
Copy link
Member

I'm biased ;) but IMO option 1 is the only one without problematic implementation issues.

If we want to provide higher-level abstractions we can do it above the RMT interface

Ok, let's just go with clock_div for now. The API of this module is anyway preliminary and we can improve it based on feedback from users.

@dpgeorge
Copy link
Member

@mattytrentini if this is ready please remove the "WIP" from the title.

@mattytrentini mattytrentini changed the title WIP: ESP32 RMT Implementation ESP32 RMT Implementation Dec 19, 2019
@dpgeorge
Copy link
Member

Squashed in to 2 commits (for implementation and docs) and merged in 0e0e613 and 7f235cb

Thanks to all involved!

Note: I renamed send_pulses to write_pulses as agreed above, and also used mp_obj_get_array() instead of the mp_obj_tuple_get()/mp_obj_list_get() pair.

@dpgeorge dpgeorge closed this Dec 20, 2019
@mattytrentini
Copy link
Contributor Author

Note: I renamed send_pulses to write_pulses as agreed above, and also used mp_obj_get_array() instead of the mp_obj_tuple_get()/mp_obj_list_get() pair.

Makes good sense - thanks!

@robert-hh
Copy link
Contributor

robert-hh commented Dec 20, 2019

Thank you for providing this module.
I just made an initial trial of that RMT class. What I am missing or misunderstanding is the option to define the 'quiet' level of the channel. As I see it now, it is always 0. But it should be the opposite of the value given in start=xx for the level of the first pulse.
And still, I would have preferred the API, where you specify both level and duration of each pulse, as the basic interface, which then can easily be used to derive the other specialized variants, like the one you have implemented.

@nevercast
Copy link
Contributor

nevercast commented Dec 20, 2019 via email

@robert-hh
Copy link
Contributor

No. According to my test, the quite level is 0. If it was the last value in the data, it should change, whether you have an odd or even number of data points. But it does not.

@mattytrentini
Copy link
Contributor Author

I just made an initial trial of that RMT class.

Thanks for trying it out!

What I am missing or misunderstanding is the option to define the 'quiet' level of the channel. As I see it now, it is always 0. But it should be the opposite of the value given in start=xx for the level of the first pulse.

For this release I wasn't able to add "idle level" which is the name Espressif use for the feature you're referring to. So, for now, the idle level can only be zero. Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

And still, I would have preferred the API, where you specify both level and duration of each pulse, as the basic interface, which then can easily be used to derive the other specialized variants, like the one you have implemented.

I'm afraid I ran out of time before I could implement it. I'm most of the way through the implementation (for all three of the tx methods) but haven't added any documentation yet and just couldn't squeeze out another couple of hours to complete it all. I'll raise it as a PR as soon as I can!

@robert-hh
Copy link
Contributor

Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

There must be a way to set the idle level, because the Pycom version supports it.

@robert-hh
Copy link
Contributor

robert-hh commented Dec 20, 2019

Setting in your code, line 101

config.tx_config.idle_level = 1;
config.tx_config.idle_output_en = true;

works as expected to set an idle level of 1. According to my tests, the second line seems not to be required, but it sounds related.
P.S.: Adding a keyword parameter idle_level=x also works. The changes to the code:
After line 74, add:

        { MP_QSTR_idle_level,                  MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, // 0 as default

Change then line 102 int0:

    config.tx_config.idle_level = !! args[3].u_int;

@mattytrentini
Copy link
Contributor Author

Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

There must be a way to set the idle level, because the Pycom version supports it.

Sorry, I chose my wording poorly; as you've already figured out, yes it's possible to set the idle level. My minor point was that, at the IDF API, the idle level isn't inferred from the data specified, instead it must be configured explicitly.
It would be possible however to infer it (same level as last value) inside the MicroPython RMT module. Though I think I prefer requiring it to be explicitly set (with the kw argument as you've suggested)...

@mcauser
Copy link
Contributor

mcauser commented Dec 22, 2019

To make it output a 38KHz carrier signal for an IR TX LED:

config.tx_config.carrier_en = 1; // was 0
config.tx_config.carrier_duty_percent = 30; // was 0
config.tx_config.carrier_freq_hz = 38000; // was 0

Should be pretty easy to parameterise these options.

Adding the carrier lets me emulate a physical remote and is detected by both IR RX 1838 LED via Saleae Logic and YS-IRTM uart module. Without the carrier, the IR TX LED is generating a signal the receivers aren't detecting.

Works on my TinyPICO running v1.12 with above carrier enable:

import esp32
from machine import Pin
r = esp32.RMT(0, pin=Pin(18), clock_div=80)
def nec(add, cmd, pulse=562):
	data = [16*pulse, 8*pulse] # 9ms leading, 4.5ms space
	zero = [pulse, pulse] # 562.5us
	one = [pulse, 3*pulse] # 562.5us, 1687.5us
	for i in range(8):
		data.extend(one if (add >> i) & 1 else zero) # address
	for i in range(8):
		data.extend(zero if (add >> i) & 1 else one) # inverse address
	for i in range(8):
		data.extend(one if (cmd >> i) & 1 else zero) # command
	for i in range(8):
		data.extend(zero if (cmd >> i) & 1 else one) # inverse command
	data.extend(zero) # not sure if this is needed
	return data
data = nec(0x00, 0x45, 562)
r.write_pulses(data, start=1)

I had to use clock_div=80 instead of clock_div=8 as 9ms is 90,000 and exceeds the 15bits for the period. 1/2 a microsecond in lost precision doesn't seem to throw the IR receivers.

@IveJ
Copy link

IveJ commented Dec 22, 2019 via email

@mattytrentini
Copy link
Contributor Author

@robert-hh (and anyone else interested) I have started on the next revision of RMT. I've added idle_level, tx_carrier settings and the other methods (same as PyCom) to specify the transmit pulses - including the double tuple variant you requested.

Sorry I couldn't get any of this done before v1.12!

I'll raise a WIP PR in the coming days.

@MrSurly
Copy link
Contributor

MrSurly commented Jan 15, 2020

@mattytrentini I'm interested -- thanks for your hard work!

@peterhinch
Copy link
Contributor

This is an awesome feature and it seems greedy to ask for more. But I will, anyway :)

@mattytrentini It would be great if it supported the carrier feature as per your revision.

To reduce allocation it would be good if .write_pulses accepted any object supporting the iterator protocol rather than being restricted to lists and tuples. Then we could use objects such as a memoryview into an array - yet it would still work with lists or tuples. Or is this unfeasible for some reason?

// At least update the method in machine_time.c.
STATIC esp_err_t check_esp_err(esp_err_t code) {
if (code) {
mp_raise_msg(&mp_type_OSError, esp_err_to_name(code));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This raises an OSError where the first arg is a string, while "normally" OSError has an integer error code as first arg. Having a string breaks code that does something like e.args[0] == ENOTFOUND. Maybe I have been doing too much socket programming lately? Shouldn't this raise an OSError with some "EOTHER" code and put the string in the second arg? Thoughts?

@pidou46
Copy link

pidou46 commented Mar 2, 2021

To reduce allocation it would be good if .write_pulses accepted any object supporting the iterator protocol rather than being restricted to lists and tuple

If write_pulse would accept an object supporting the iterator protocol like peterhinch have suggested, I could write a generator function that allow me to adjust pulses pattern from a asyncio coro on the fly.
This would allow me to sync RMT pulse to other signal on the fly.

This would make RMT very powerfull and pythonic

@mattytrentini
Copy link
Contributor Author

@pidou46 My first design accepted an iterator...but then I ran into a problem. The IDF API requires a C-style array to be passed to it - so an iterator offers little benefit and complicates the implementation.

@nevercast
Copy link
Contributor

Perhaps a bit of python wrapping code could chunk up an iterator into blocks of arrays as required by the C api. Matt would have to essentially do that anyway in the driver and there is little reason to add the extra complexity. Perhaps @pidou46 or @peterhinch could experiment with something that consumes an iterator/generator and sends that to RMT?

@peterhinch
Copy link
Contributor

My concern about allocation would be addressed by supporting array instances, ideally integer, half word, and bytearrays. Is this problematic?

@pidou46

I could write a generator function that allow me to adjust pulses pattern from a asyncio coro on the fly. This would allow me to sync RMT pulse to other signal on the fly.

You are looking for support for concurrency. This would be very nice but would require an API enhancement to enable a nonblocking wait on completion or polling of completion status. This would enable the calling code to prepare a data array while its predecessor was being transmitted.

The Pythonic approach would be to support uasyncio. The RMT API would need to be enhanced, ideally so that it could be configured as a StreamWriter object. uasyncio would then poll it using the uselect.poll mechanism and feed it more data when the RMT is ready to accept it. However any such solution would need to consider uasyncio latency: this can be tens of ms. To avoid gaps in the data stream, the RMT would need to implement a buffer.

The C code would take the data from the buffer, put it in a C array and feed it to the IDF API. It would then flag readiness to receive data via the ioctl method and the buffer would be refilled.

Such uasyncio support would enable the output of arbitrary pulse trains of lengths to infinity. (And beyond...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.