esp32: ADC and ULP improvements #16521

purewack · 2025-01-03T22:53:16Z

Summary

I have updated the ADC and ULP modules to fix an issue with the RISCV adc function causing a conflict of drivers (as mentioned in my PR #16469), preventing my application from working.
I have also used the work contributed by #11572 (Thanks @ThinkTransit ) and extended it to suit it for the new way of building the firmware, as that PR is quite outdated now.

ADC

I have started by updating the ADC to use the new driver (esp_adc/adc_oneshot.h).
The reason for this is that when using custom c modules, and the need to call the ulp_adc_init() function from the esp-idf arises, an error ADC: CONFLICT! driver_ng is not allowed to be used with the legacy drive was displayed at runtime, but compilation was still successful.

There are several errata notes about not being able to change the bit-width of the ADCs certain chips.
The only chip that can switch resorption to a lower one is the normal esp32.
esp32 C2 & S3 are stuck at 12 bits, while S2 is at 13 bits.
On the S2, you can change the resolution, but it has no effect on the resolution, rather, it prevents attenuation from working at all!
I have set the resolution to the be maximum possible for each SoC, with the esp32 being the only one not throwing errors when trying to set the bit-width to 9,10,11 or 12 bits using ADC.width(bits)

espressif/esp-idf#13128
https://github.com/espressif/esp-idf/blob/release/v5.2/components/soc/esp32s3/include/soc/soc_caps.h
https://docs.espressif.com/projects/esp-chip-errata/en/latest/esp32s2/03-errata-description/index.html
https://docs.espressif.com/projects/esp-idf/en/v4.2/esp32/api-reference/peripherals/adc.html

ULP

To feel the benefits of the new ADC driver, I have in turn extended and improved the functionality of the ULP module.
I went with the following strategy:

Enable RISCV ULP for chips that support it by default (S2 & S3)
Keep existing ULP functionality by using FSM where RISCV is not available
Give users the option to recompile the firmware with a preference for FSM where RISCV is enabled by default

It is now also possible to compile in the binary image of the ULP code, by doing the following:

Create or modify a board definition and add a ulp folder for ULP source code
When this is done, edit mpconfigboard.cmake and add the following:

set(ulp_embedded_sources ${MICROPY_BOARD_DIR}/ulp/ulp_source_name.c)
# or
set(ulp_embedded_sources ${MICROPY_BOARD_DIR}/ulp/ulp_source_name.S)

If there are any custom c modules which want to use the generated ulp_embedded.h header, add:

set(ulp_dependants ${MICROPY_BOARD_DIR}/cmodules/module_name.c)

After compilation, the ULP module will have variables (ulp code variables prefixed with var_) exposed as the modules is an automatic dependent.
The ULP module will then have an extra function called run_embedded() which does what is says on the tin.

Users then can load & run the code using ULP.run_embedded() and interact with the code using ULP.read(address) and/or ULP.write(address, value).

Users will have to option to compile for FSM preference by adding the lines:

set(SDKCONFIG_DEFAULTS
    ...
    boards/sdkconfig.ulp_fsm
)


set(PREFER_FSM_ULP ON)

to their mpconfigboard.cmake file.

I believe this is an elegant solution to this problem, as FSM and RISCV components are mutually exclusive in the esp-idf.

I also added missing functions such as

ULP.pause()
ULP.resume()
for pausing and resuming ULP execution (in reality it only stops the wakeup timer)

RTC options for ULP

I also ran into a number of issues with how the ULP code expects the main app to set up its IO.
From my testing, FSM has no way to set pins into RTC mode, therefore no outputs can be controlled by the ULP.
RISCV has functions to do that from within the ULP, but FSM does not seem to have it so easy.

I have added the following methods to the ULP class:

ULP.rtc_init(pin_number)
ULP.rtc_deinit(pin_number)
ULP.adc_init(unit, channel)

These functions allow setting up the pins for IO, as well as setting up the ADC correctly for the ULP to sample away with!

Example usage of embedded app:

ulp/main.c:

#include <stdint.h>
#include "ulp_riscv_gpio.h"
#include "ulp_riscv_utils.h"
#include "ulp_riscv.h"

#define LED_GPIO_NUM 3

unsigned int var_counter = 0;
unsigned int var_count = 1;

void main(){
      if(var_count){
          var_counter++;
      }
      ulp_riscv_gpio_output_level(LED_GPIO_NUM, 1);
      ulp_riscv_gpio_output_level(LED_GPIO_NUM, 0);
}

The app is then compiled into the firmware by modifying, for example the GENERIC_S3 board.

main.py:

import time
import esp32

ulp = esp32.ULP() #default wakeup period is 10ms
ulp.rtc_init(3)
ulp.run_embedded()

time.sleep(1)
print( ulp.read(ulp.VAR_COUNTER) ) # prints for example 325
ulp.write(ulp.VAR_COUNT, 0)

time.sleep(1)
print( ulp.read(ulp.VAR_COUNTER) ) # still prints 325, as the counter was stopped in the ulp code

Example usage loading ADC sampler:

ulp/main.c:

#include <stdint.h>
#include "ulp_riscv_adc_ulp_core.h"
#include "ulp_riscv_utils.h"
#include "hal/adc_types.h"

unsigned int var_reading;

void main(){
      var_reading = ulp_riscv_adc_read_channel(ADC_UNIT_1, ADC_CHANNEL_1);
}

The app is then compiled into the firmware by modifying, for example the GENERIC_S3 board.

main.py:

import time
import esp32

ulp = esp32.ULP() 
ulp.adc_init(0,1) # ADC1, Channel 1, GPIO 2 on esp32 S3

with open("riscv,bin","rb") as file:
    bin = file.read()
ulp.run(bin)

var_reading = 0x500000dc # sourced from building the ulp firmware using the generated `.ld` file
print( ulp.read(var_reading) ) # print raw adc reading, resolution dependent on chip

Testing

I have gone through a lot of builds and different combinations, all seem to work fine.
✅ esp32 with FSM embedded code
✅ loading test.bin on esp32 FSM (requires RTC pin setting)
✅ running esp32 s3 RISCV embedded code
✅ loading riscv.bin by first compiling embedded code, then with no embedded app on esp32 S2
✅ using the build ULP binary or an S3 and running it on another S3 that was build with no embedded binary

Test result of running a simple pin flip with no delays; FSM in an infinite loop on ESP32:

Test result of running a simple pin flip with delays in an infinite loop; RISCV on ESP32 S3:

I have included in my example branch two modified board definitions which include everything that I have used to test and confirm functionality of the ULP.

ESP32_GENERIC
ESP32_GENERIC_S3

Both include their respective ULP embedded code, along side a ulp_example_fsm and ulp_example module for ease of testing.
Simply build, flash, and run import ulp_example

The test increments a counter variable, reads ADC, and quickly toggles a pin on and off.

Side-effects

During testing of this ULP code:

#include <stdint.h>
#include "ulp_riscv_gpio.h"
#include "ulp_riscv_utils.h"
#include "ulp_riscv.h"

// GPIO number for the LED
#define LED_GPIO_NUM 3

// Delay value (adjust for desired blinking speed)
#define DELAY_CYCLES 200000

unsigned int var_counter = 0;
unsigned int var_count = 1;

void main(){
    // Configure the GPIO as an output
    ulp_riscv_gpio_init(LED_GPIO_NUM);
    ulp_riscv_gpio_output_enable(LED_GPIO_NUM);

    while (1) {
        if(var_count){
            var_counter++;
        }
        // Toggle the GPIO
        ulp_riscv_gpio_output_level(LED_GPIO_NUM, 1);
        ulp_riscv_delay_cycles(DELAY_CYCLES);
        ulp_riscv_gpio_output_level(LED_GPIO_NUM, 0);
        ulp_riscv_delay_cycles(DELAY_CYCLES);
    }
}

I was not able to reference the var_counter and var_count variables. They were compiled out as the function could never return.
As another side effect of an infinite loop, you cannot stop the code by calling ULP.pause() because the function never returns to be then woken up by the timer.

Trade-offs and Alternatives

I believe the RISCV ULP has matured enough to be useful to a number of users, as it is far simpler code rather than being stuck writing assembly for the FSM.

Add the ability to disable the built in ULP module using mpconfigport.h Add the ability to compile custom ULP image into the build by defining the cmake variables: ulp_app_name ulp_sources ulp_exp_dep_srcs Signed-off-by: Damian Nowacki (purewack) <bobimaster15@gmail.com>

ThinkTransit · 2025-01-04T13:15:14Z

Nice work @purewack I will close my unfinished PR as I think you have covered everything in mine?

Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

stephanelsmith · 2025-01-07T01:58:11Z

This looks fantastic. I can't wait to try it out! Greatly appreciate your pioneering efforts on this front!

esp32/adc: Updated the module to use new driver (esp_adc). Tested and confired working on: - esp32 - esp32 C3 - esp32 S2 - esp32 S3 esp32/ulp: Enable RISCV ULP by default on targets which support it esp32 S2 & S3 have an option of reverting back to using the FSM in a custom build. Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

dpgeorge · 2025-01-23T06:17:29Z

Thanks for the contribution, this does look very good!

One initial comment/question: it looks like ULP.load_binary() is gone, and now ULP.run() both loads and starts the ULP program. This is a breaking change to the API.

Is this a necessary change? Is there a use-case where the ULP program is loaded just once and then run many times? And possibly started some time after loading?

Also, are the ADC changes to use the new driver independent to the ULP improvements? If so, it would be beneficial to move those changes (hopefully just cherry-pick a commit or two from here) to a separate PR, to make it easier to review and faster to merge.

purewack · 2025-01-23T10:35:50Z

Hi @dpgeorge !

Thanks for looking through the PR.

What is the best approach for this now? I'm new to all this as I have only ever worked on my private repos before.

Should I close this PR and open separate ADC and ULP ones ?

Regarding the ULP breaking change, I can revert the functionality back to separate load and run functions. The reason I merged them into one was I didn't see a use case where a user might load a binary to them later execute it as opposed to just calling load and run in a quick succession, it felt like an extra step to do the same thing.

I'm am happy to do what is required in order not to break anything, but some pointers would be appreciated 😅

dpgeorge · 2025-01-23T11:22:00Z

What is the best approach for this now?

The work you've done here looks really good, thank you. I think it's probably 90% ready to merge. It's just the last 10% can take some time to go back and forth.

Considering there are ULP API changes which need a bit of discussion/work, it would be best to split out the ADC changes as a separate PR, as I think they can get merged relatively quickly. Then leave this PR as-is until the ADC is merged, then we'll pick it up again.

purewack · 2025-01-23T22:48:35Z

I have opened PR #16630 and singled out all the ADC changes there @dpgeorge

purewack · 2025-01-25T18:11:10Z

I've included RTC pin functionality in the changes here too.
I think given what I have learned now, it would be a good idea to extend either the RTC module or the Pin class itself to allow setting up pins in RTC mode so that the ULP can control the outputs / inputs.

The question is which approach is more appropriate: RTC or Pin class?
I think this would be the next step to focus on in terms of breaking up this PR.

purewack · 2025-02-13T11:40:09Z

Ok some time has passed since the last time this was spoken of. I have made all the necessary changes (and corrected my mistakes @dpgeorge) to the ADC for the ULP module to work with riscv on the PR #16630.

I think what I will do is restore the functions reverting any breaking changes I have introduced to the ULP module, but extend the module to both add missing functions like pause and resume while also adding embedded firmware specific functions like they are now.

Regarding the RTC pin functionality I am unsure whether is is best to keep them here or place them in a different module, but as was mentioned earlier this will require further discussion and some back and forth in this regard as well.

projectgus

This is a nice clean implementation and I think it'll be really handy to be able to write small RISC-V programs and embed them in MicroPython.

I have been thinking a lot about supporting custom builds with an embedded RISC-V program as the "best" option for users, mostly the trade-off of functionality to complexity. The more "MicroPython-y" way would be to load the ULP binary at runtime. IIUC the trade-offs are:

There is a temporary RAM cost to loading the binary from the filesystem. That'll be true until "ROMFS" is fully supported, but also the binary is small.
Generating the var_ variable entries is a really nice bit of functionality, and it'd be more work to bring this over to dynamically loaded ULP programs. Although you could imagine a relatively simple tool that takes a ULP .elf file as input and outputs some kind of .mpy file that embeds the variable constants and the binary code...

To be clear, I'm still OK with merging support for embedding the RISC-V ULP binaries and you should keep it in this PR for now. I am curious what you think about the good/bad aspects of an alternative approach, though?

projectgus · 2025-03-04T00:33:28Z

docs/library/esp32.rst

-.. warning::
+.. note::
+    By default RISCV is preferred on supported chips, however, you can configure a custom build 
+    of micropython to revert back to using the FSM ULP.


Suggested change

of micropython to revert back to using the FSM ULP.

of MicroPython to revert back to using the FSM ULP.

In addition to the nitpick about capitalisation, this is something of a breaking change for anyone who was using the ULP class on S2 and S3 before, yes? Although I agree the RISC-V one is much nicer to work with, we might have to defer swapping defaults until MP 2.0 (or add a build variant for some boards).

It's kind of a shame that ESP-IDF doesn't support enabling both ULP types... I'm guessing you looked into this and there's no way to make it work?

Unfortunately, yes I have looked deep into it and it looks like they are mutually exclusive so we only get to pick one or the other at build time :/ Maybe in the future IDF releases there will be an option for runtime selection as I don't know if this is a hardware of software limitation of the ESP32.

projectgus · 2025-03-04T00:38:32Z

docs/library/esp32.rst

+.. note::
+   Exposed variables are relative address offsets from RTC_SLOW_MEM address, 
+
+   however you can use full address values also, if loading ULP code from outside the compilation process.

 .. class:: ULP()


Even keeping the API (and the implementation) mostly the same, what do you think about renaming the class to RVULP if the build is configured for RISC-V?

Otherwise it's quite confusing to take Python code that works on one board and find it's valid Python on another board, but also doesn't work.

This also makes it easier to document some of the differences (like ULP.set_wakeup_period) between the two implementations.

I agree and this was going through my head after implementing the changes. This change would be very breaking if the RISCV preference was a default on future builds. I agree the RVULP class is a great idea!

projectgus · 2025-03-04T00:41:00Z

ports/esp32/esp32_ulp.c

@@ -28,14 +28,33 @@

 #if CONFIG_IDF_TARGET_ESP32 || CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3

+#if CONFIG_IDF_TARGET_ULP_PREFER_FSM || CONFIG_IDF_TARGET_ESP32
+    #define TYPE_RISCV 0


Nitpick, we don't indent inside #if blocks in MicroPython.

projectgus · 2025-03-04T00:45:30Z

ports/esp32/esp32_ulp_qstr.py

+"""
+This script reads in the given CMSIS device include file (eg stm32f405xx.h),
+extracts relevant peripheral constants, and creates qstrs, mpz's and constants
+for the stm module.


This docstring (and the block comment below) need updating for this context.

purewack · 2025-03-04T10:12:57Z

I have started a thread regarding this PR: https://github.com/orgs/micropython/discussions/16819#discussioncomment-12380769 to try and get more opinions on the breaking changes but now that @projectgus has suggested the RVULP class, I think it is the solution for this.

edit:
Ok looks like there is hope for enabling the FSM and RISCV at the same time? adafruit/esp-idf#16. Adafruit has done some work for their CircuitPython port so It looks like it could be possible if officially resolved by Espressif

purewack · 2025-03-05T00:09:30Z

I've been doing a lot of work on my own board running micropython (esp32s3) and have learned a great deal about how things work under the hood. I've been using the ULP changes I have made, and noticed that sometimes my ULP code will not start (still trying to determine why, happens when imported using a wrapper module that actually interfaces with the ULP using VAR_*).

I needed to first pause then resume the timer (pause halts the program too for RISCV) to get the binary to work after importing my wrapper module in a different script for example, as I have the wrapper frozen into the firmware.

Main question:
From my understanding the ULP is a singleton object.
Would it benefit from a finaliser to stop the running ULP code upon GC or explicit del ?

I think the code should not run, especially after soft-reset, which it does, and always did even before the changes I had made.

This can be easily confirmed by writing a counter program for the ULP, running that binary then doing a soft-reset, and finally reading the counter value from RTC memory, observing that it has not reset, and is still increasing.

import esp32
u = esp32.ULP()
u.run_embedded() # or any other way using open() and ulp.load() etc
u.read(u.VAR_COUNTER)

# soft reset here
import esp32
u = esp32.ULP
u.read(u.VAR_COUNTER)

projectgus · 2025-03-17T23:55:44Z

From my understanding the ULP is a singleton object.
Would it benefit from a finaliser to stop the running ULP code upon GC or explicit del ?

Sorry, I missed this question when you asked it!

The most common pattern for these "hardware-adjacent" singletons is to add a deinit() function in the soft reset path of the main function (look for label soft_reset_exit: in ports/esp32/main.c). This is more deterministic than a finaliser, and has less overhead when you know you're going to clean up at most one of something.

However, there are some low level board functions (like the network connection) that remain running over a soft reset. I feel like conceptually ULP is closer to this, because you can even (for example) go into deep sleep and have the ULP keep running.

So maybe the cleanup code won't need to stop the ULP program running, but should still clean-up any Python-side state about the ULP so that it stays "safe" after the reset cycle when the code can re-initialise it and determine the current state? I'm not sure if that idea makes sense, let me know if it doesn't.

purewack mentioned this pull request Jan 3, 2025

esp32: Add the ability to disable ULP module using mpconfigport.h and ULP images built into custom builds #16469

Closed

purewack marked this pull request as draft January 4, 2025 20:38

esp32: ADC driver update to the new esp_adc.

2545b28

Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

purewack force-pushed the master branch 3 times, most recently from 41e871c to a830946 Compare January 5, 2025 01:14

purewack added 3 commits January 5, 2025 01:20

esp32/ulp: Set ULP type to RISCV by default if supported.

a7dbd33

Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

esp32: ULP read and write RTC memory.

50e66d6

Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

esp32: ULP RTC io options.

13a7877

Signed-off-by: Damian Nowacki (purewack) bobimaster15@gmail.com

purewack force-pushed the master branch from a830946 to dedcd13 Compare January 5, 2025 01:21

purewack marked this pull request as ready for review January 5, 2025 01:26

projectgus self-requested a review January 6, 2025 23:55

purewack force-pushed the master branch from dedcd13 to ef88c1a Compare January 10, 2025 22:44

purewack force-pushed the master branch from ef88c1a to deba6f1 Compare January 10, 2025 22:55

dpgeorge added the port-esp32 label Jan 23, 2025

purewack mentioned this pull request Jan 23, 2025

esp32: ADC driver update to the new esp_adc. #16630

Open

projectgus reviewed Mar 4, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

esp32: ADC and ULP improvements #16521

esp32: ADC and ULP improvements #16521

purewack commented Jan 3, 2025 •

edited

Loading

ThinkTransit commented Jan 4, 2025

stephanelsmith commented Jan 7, 2025

dpgeorge commented Jan 23, 2025

purewack commented Jan 23, 2025

dpgeorge commented Jan 23, 2025

purewack commented Jan 23, 2025

purewack commented Jan 25, 2025

purewack commented Feb 13, 2025

projectgus left a comment

projectgus Mar 4, 2025

projectgus Mar 4, 2025

projectgus Mar 4, 2025

purewack Mar 4, 2025

projectgus Mar 4, 2025

purewack Mar 4, 2025

projectgus Mar 4, 2025

projectgus Mar 4, 2025

purewack commented Mar 4, 2025 •

edited

Loading

purewack commented Mar 5, 2025

projectgus commented Mar 17, 2025

	of micropython to revert back to using the FSM ULP.
	of MicroPython to revert back to using the FSM ULP.

esp32: ADC and ULP improvements #16521

Are you sure you want to change the base?

esp32: ADC and ULP improvements #16521

Conversation

purewack commented Jan 3, 2025 • edited Loading

Summary

ADC

ULP

RTC options for ULP

Example usage of embedded app:

Example usage loading ADC sampler:

Testing

Side-effects

Trade-offs and Alternatives

ThinkTransit commented Jan 4, 2025

stephanelsmith commented Jan 7, 2025

dpgeorge commented Jan 23, 2025

purewack commented Jan 23, 2025

dpgeorge commented Jan 23, 2025

purewack commented Jan 23, 2025

purewack commented Jan 25, 2025

purewack commented Feb 13, 2025

projectgus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

purewack commented Mar 4, 2025 • edited Loading

purewack commented Mar 5, 2025

projectgus commented Mar 17, 2025

purewack commented Jan 3, 2025 •

edited

Loading

purewack commented Mar 4, 2025 •

edited

Loading