This might be fixed by #7163.

#7163 fixes it only for devices with SPIRAM, because it will then use SPIRAM for the heap.

Still no progress :-(
Adding reference to #6785, an interesting extension of this.

@dpgeorge @robert-hh I'd like to make a PR, but I like to get some comments on proposed heap size calculation.

This issue specifically mentions ESP32-S2, but I think we need to start with plain ESP32 for which:

1. on older IDF version (eg v4.0.2, with ESP32), the amount of MicroPython heap is 111168 bytes, and the IDF is left with 88502 bytes
2. recent IDF versions on ESP32 give only 64000 MicroPython heap (as mentioned above) and already break existing user code, because that code expects around 110k of heap

So there's already something there to fix, just dealing with different IDF versions.

The things that make it difficult are:

1. different IDF versions have a different layout for the heap, some have more contiguous heap than others
2. different IDF versions use more or less heap for their own internal operations
3. different chip variants (S2, S3, C3, etc) have different amounts of total memory
4. different user applications require different amounts of memory for MicroPython heap vs internal things (like wifi buffers, BLE)

To solve (1) we need #3580, then the MicroPython heap can take as much or as little heap as it likes, it doesn't matter how much it's fragmented within the IDF (to an extent).

To solve (4) we need a way for the user to specify the amount of heap to allocate, eg #6785. An alternative to #6785 would be to create an initial small heap (say 8k) which is used to run boot.py, and within boot.py the user must allocate a second, larger heap segment, eg esp32.allocate_heap(50000). Then both heaps are available from then on.

To solve (2) and (3) we need a sensible default value (or ratio) for MicroPython heap vs IDF free heap vs IDF used heap (already used by the time MicroPython starts). As proposed above by @tom-van, the value of 1/3 for MicroPython heap out of total heap does seem like a good sensible value, although it does give less heap than the original 111168 that ESP32 used to have.

A lot of this was already mentioned in #7963, so I guess this is just a summary of things.

We don't need to solve everything now. I think the idea proposed above of 1/3 is very reasonable and a good step forward.

In 23b1a4e I made it so that MicroPython allocates at most 1/2 of the total IDF heap. This makes sure 111168 bytes are still available on standard ESP32 with IDF 4.2.

@dpgeorge thanks for addressing the issue!

I confirm that 23b1a4e makes webrepl working on GENERIC_S2
Unfortunately IDF heap on S2 is still too small for using TLS, so e.g. upip fails:

>>> import upip
>>> upip.install('logging')
Installing to: /lib/
Error installing 'logging': [Errno 12] ENOMEM, packages may be partially installed

The good point the problem is correctly reported, no cryptic WiFi disconnect happens this time.
Surprisingly upip install goes well on GENERIC_C3

Just for those who want TLS functionality on S2 now I refer to fine tuned heap allocation I was working on
tom-van@07c1ef8
Sorry, due to lack of time I was able to test it only on ESP32 and S2, IDF 4.0.2 and 4.4

I'm looking forward to some dynamic way to partition the heap.