Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:   +32 +0.004% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@projectgus, I built micropython for RPI_PICO_W with your changes and tested them.
Looks great.

As part of that, I ran the attached example running a periodic Timer once per second during the lightsleep.
lightsleep_with_timer.txt

I just happened to have this code lying around for testing sleep_ms and I thought to give it a try.
This specific case worked correctly.
lightsleep() returned after 1 second as expected and the Timer function ran.

I know interrupts from other sources (EG: Pin) don't work but this one does.

Thanks for taking a look and testing this, @cpottle9!

I just happened to have this code lying around for testing sleep_ms and I thought to give it a try. This specific case worked correctly. lightsleep() returned after 1 second as expected and the Timer function ran.

Ironically, machine.Timer can now wake the rp2 from lightsleep because it uses the alarm pool (timer interrupt 3). So I assume it doesn't work on either V1.23 or V1.24, and may stop working on future versions if we switch back to also using the alarm pool for the soft timer!

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.57%. Comparing base (a9945fc) to head (e9b8a36).
Report is 265 commits behind head on master.

@@           Coverage Diff           @@
##           master   #16431   +/-   ##
=======================================
  Coverage   98.57%   98.57%           
=======================================
  Files         164      164           
  Lines       21349    21349           
=======================================
  Hits        21045    21045           
  Misses        304      304           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Having a constant lwIP tick counter of 64ms is probably a waste when it will tell us when it next needs to wake up via sys_timeouts_sleeptime.

@projectgus and other maintainers of the rp2 port, I've been following changes related to lightsleep and the shoe just dropped for me. I did not see what you are trying to do until now.

You are trying to solve world hunger for lightsleep and I don't think that is good use of your time.
By that, I mean you are trying to provide maximum power reduction and yet not break functionality.
I see the functionality you are trying to preserve includes USB, WIFI, and use of the second core.

If I'm running a PICO_W from battery to save power the first thing I need to do is disable the cyw43 (and WIFI) when I don't need it. That includes not using the system LED. I have an application where I do wlan.deinit() and then lightsleep.
Further, when my application is deployed I don't need USB (I know, you fixed that one).
Running on battery I probably don't want to make use of the second core either.

I propose you need make lightsleep simpler and empower developers(like me) to decide which hardware blocks need to enabled for a given application.
To that end, I created micropython code that allows a developer to modify the WAKE_ENx and SLEEP_ENx registers on the RP2040 and RP2350. If you have time, take a look at my github RP2-PowerControl.
My idea is users stop using machine.lightsleep and use time.sleep_ms instead ( make machined.lightsleep do the same thing as time.sleep_ms).
Whenever a __WFI or __WFE happens the contents of SLEEP_ENx registers will be used.
Further, if a user knows they do not need specific blocks the can update WAKE_ENx registers to power them down all the time.

Having a constant lwIP tick counter of 64ms is probably a waste when it will tell us when it next needs to wake up via sys_timeouts_sleeptime.

Ah, thanks @peterharperuk! The rp2 port is the only tickless port in MicroPython at the moment, so it's the only place this really counts. If we can easily make lwIP tickless as well then we have a lot more options. I'd assumed this wasn't simple but sounds like it may well be.

You are trying to solve world hunger for lightsleep and I don't think that is good use of your time.
By that, I mean you are trying to provide maximum power reduction and yet not break functionality.
I see the functionality you are trying to preserve includes USB, WIFI, and use of the second core.

This is a good point, @cpottle9, although it's less "not break functionality" and more "not crash or hang" which is the bottom line here. For example, the reason that USB gets special handling in rp2 lightsleep is because disabling (or suspending) on sleep and re-enabling on wake caused intermittent crashes when either the peripheral or TinyUSB gets in some permanent bad state. Similarly, the workarounds for the second core are to prevent execution stalling indefinitely if the other CPU gets permanently stuck in WFI. If the impact is limited to "feature stops on sleep and starts on wake" or "power consumption is higher when this feature is in use" then these are probably reasonable trade-offs. Permanent hangs or hard faults aren't reasonable trade-offs.

I propose you need make lightsleep simpler and empower developers(like me) to decide which hardware blocks need to enabled for a given application.
To that end, I created micropython code that allows a developer to modify the WAKE_ENx and SLEEP_ENx registers on the RP2040 and RP2350. If you have time, take a look at my github RP2-PowerControl.

That's a great project, and I think it's a good approach for people who want the absolute most control over the sleep modes.

Part of the goal of MicroPython, and the machine module in particular, is to provide some abstraction of the hardware. To date, low-power modes are not the best example of this in MicroPython as there's not much of a common design and things that are quite different between each port. However, long term that's the kind of direction we'd want to go. I would say that as a first goal, having a "lowest reasonable consumption" machine.lightsleep() and a "low power but max functionality" time.sleep() will meet a lot of people's needs.

That's not to say that something like RP2-PowerControl isn't a great option for folks who need that fine-grained register-level control, though.

If the impact is limited to "feature stops on sleep and starts on wake" or "power consumption is higher when this feature is in use" then these are probably reasonable trade-offs

I realise there is another trade-off we could choose for some of these states, which would be to have lightsleep degrade to a normal WFI or even raise a RuntimeException if we know it's not safe to disable clocks for lightsleep.

There are still footguns we don't want to leave lying around, though. For example, even if someone's battery powered application isn't designed to run with USB in the field then they might want to sometimes plug USB in and do some debugging. If the behaviour in that state is significantly different then it can become quite hard to debug.

@projectgus, your USB issue is valid.
But dealing with can be the developers problem not yours.

Their application can determine if USB is present and have RP2-PowerCtrl disable USB only when it is not.

By the way, I initially coded RP2-PowerCtrl as part of modrp2.c.
If the owners of micropython agree I could finish that work and create a pull request.

@projectgus one other thing. Prior to commit d1423ef on entry lightsleep saved the current value of sleep_en0 and sleep_en1.
Before exiting it restored the saved values.

Now before exiting lightsleep sets those registers to the power on defaults.

That means if someone uses RP2-PowerCtrl and then calls lightsleep their changes to sleep_en0 and sleep_en1 are lost.

Should I open an issue?

By the way, I initially coded RP2-PowerCtrl as part of modrp2.c.
If the owners of micropython agree I could finish that work and create a pull request.

That could be one way forward, although if everything works the same in pure Python code then it's less appealing to add binary size for it in modrp2. I also just tagged you in this discussion, where I vaguely outlined another possible approach:
#16183 (comment)

Now before exiting lightsleep sets those registers to the power on defaults.

That means if someone uses RP2-PowerCtrl and then calls lightsleep their changes to sleep_en0 and sleep_en1 are lost.

Should I open an issue?

Yes, we can definitely change this back. If possible then suggest you open a Pull Request restoring behaviour that works for RP2-PowerCtrl, and then we can go from there.

Couple of things have become apparent about this approach:

1. It seems much more sensible to merge Alarm pool sleep changes #16454 and go back to using the Pico alarm pool. That PR currently switches to use a dedicated timer alarm for lightsleep wakeup, which seems like a good choice but it will mean that the lwIP tick timer still wakes us up from lightsleep.
2. Switching to "tickless" lwIP doesn't immediately solve the problem either. LWIP runs all its "cyclic" timers all of the time, the fastest of which run at 100ms for IGMP and MLD6. MicroPython will need to either calculate when no interface is enabled and disable timer callbacks (which means lightsleep+wifi can work seamlessly), or we disable either the lwIP timer callbacks while we're in lightsleep or the entire alarm pool (in which case we may as well keep the 64ms lwIP tick).

Quick update after discussing with @dpgeorge : Strong preference for #16454 to be the long-term fix. Hopeful that there will be a pico-sdk release with the final alarm pool fix included and we can go direct to that one.

So I'm not planning to pick this approach up again for now. If this is a major inconvenience for anyone, please leave a comment.

EDIT/note-to-self: If this is eventually closed then should open a new PR with just the test improvements.

@projectgus, thanks for the update. You prompted me to look at #16454.
It explains the symptoms I described in #16562.

If I understand correctly it will be a while before the issues in pico-sdk are addressed.
So, this problem will not get resolved quickly. Correct?

If I understand correctly it will be a while before the issues in pico-sdk are addressed. So, this problem will not get resolved quickly. Correct?

The issue in pico-sdk is addressed now, but there hasn't yet been a pico-sdk release that includes the fix. Once a release comes out, and assuming no other issues crop up when we update pico-sdk, then we'll be able to proceed with the changes in #16454.

Closing in favour of the approach from #16454