Skip to content

Try to rehabilitate circuitpython appveyor build #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

jepler
Copy link
Owner

@jepler jepler commented Mar 31, 2018

.. but it didn't work out, leaving 11 test failures or so that I have no interest in chasing down. I create this PR just so the code is floating out there in the universe for some interested person to find.

jepler added 6 commits March 31, 2018 10:58
.. on other platforms, alloca can be used as an alternative.
(micropython uses m_new + m_del but this seems unnecessary as alloca
is already required by micropyton)
(different tricks may be required to fully avoid inlining of these
functions on other compilers, however)
.. unsetting this causes python3 to fail to start at all,
stating "Fatal Python error: Failed to initialize Windows random API
(CryptoGen)".  See https://bugs.python.org/issue20614
jepler pushed a commit that referenced this pull request Sep 10, 2018
jepler pushed a commit that referenced this pull request Oct 21, 2018
jepler pushed a commit that referenced this pull request Nov 22, 2018
jepler pushed a commit that referenced this pull request Jul 13, 2019
jepler pushed a commit that referenced this pull request Jul 13, 2019
jepler pushed a commit that referenced this pull request Jul 13, 2019
jepler pushed a commit that referenced this pull request Jul 13, 2019
jepler pushed a commit that referenced this pull request Jul 13, 2019
jepler pushed a commit that referenced this pull request Nov 22, 2019
Removed robohatmm1_m0 from build.
jepler added a commit that referenced this pull request Nov 25, 2019
It's probably not the whole story, however, this fixes a crash observed
when bulk copying data to an nRF board using `dd`.

Basically, the call stack looked like this when resetting into safe mode:
    #0 reset_into_safe_mode reason=reason@entry=GC_ALLOC_OUTSIDE_VM
    #1 gc_alloc
..  #4 external_flash_write_block
.. adafruit#11 usb_background
   adafruit#12 run_background_tasks
   adafruit#13 common_hal_neopixel_write
.. adafruit#18 start_mp

i.e., during early startup, it is not okay yet to call allocation functions
like m_malloc_maybe that use the garbage collected heap.  However,
nRF's neopixel_write (which already includes special handling to avoid
heap allocations for the status pixel!) can enter background tasks, which
do nearly arbitrary things including heap allocations.

We re-use the same test that switches from heap allocation to stack
allocation for the pattern buffer.
jepler pushed a commit that referenced this pull request Dec 17, 2019
jepler pushed a commit that referenced this pull request Jan 3, 2020
stm32 explicitily enable/disable vbus sense, fix warning
jepler pushed a commit that referenced this pull request Feb 11, 2020
update to adafruit master
jepler pushed a commit that referenced this pull request Feb 25, 2020
jepler pushed a commit that referenced this pull request Mar 16, 2020
@jepler jepler closed this Mar 17, 2020
jepler pushed a commit that referenced this pull request May 13, 2020
jepler pushed a commit that referenced this pull request May 13, 2020
jepler pushed a commit that referenced this pull request May 15, 2020
jepler pushed a commit that referenced this pull request Jun 5, 2020
Merge and a few final tweaks
jepler pushed a commit that referenced this pull request Jul 28, 2020
jepler added a commit that referenced this pull request Aug 30, 2020
A crash like the following occurs in the unix port:
```
Program received signal SIGSEGV, Segmentation fault.
0x00005555555a2d7a in mp_obj_module_set_globals (self_in=0x55555562c860 <ulab_user_cmodule>, globals=0x55555562c840 <mp_module_ulab_globals>) at ../../py/objmodule.c:145
145	    self->globals = globals;
(gdb) up
#1  0x00005555555b2781 in mp_builtin___import__ (n_args=5, args=0x7fffffffdbb0) at ../../py/builtinimport.c:496
496	                mp_obj_module_set_globals(outer_module_obj,
(gdb)
#2  0x00005555555940c9 in mp_import_name (name=824, fromlist=0x555555621f10 <mp_const_none_obj>, level=0x1) at ../../py/runtime.c:1392
1392	    return mp_builtin___import__(5, args);
```

I don't understand how it doesn't happen on the embedded ports, because
the module object should reside in ROM and the assignment of self->globals
should trigger a Hard Fault.

By checking VERIFY_PTR, we know that the pointed-to data is on the heap
so we can do things like mutate it.
jepler added a commit that referenced this pull request Oct 1, 2020
It was incorrect to NULL out the pointer to our heap allocated buffer in
`reset`, because subsequent to framebuffer_reset, but while
the heap was still active, we could call `get_bufinfo` again,
leading to a fresh allocation on the heap that is about to be destroyed.

Typical stack trace:
```
#1  0x0006c368 in sharpdisplay_framebuffer_get_bufinfo
#2  0x0006ad6e in _refresh_display
#3  0x0006b168 in framebufferio_framebufferdisplay_background
#4  0x00069d22 in displayio_background
adafruit#5  0x00045496 in supervisor_background_tasks
adafruit#6  0x000446e8 in background_callback_run_all
adafruit#7  0x00045546 in supervisor_run_background_tasks_if_tick
adafruit#8  0x0005b042 in common_hal_neopixel_write
adafruit#9  0x00044c4c in clear_temp_status
adafruit#10 0x000497de in spi_flash_flush_keep_cache
adafruit#11 0x00049a66 in supervisor_external_flash_flush
adafruit#12 0x00044b22 in supervisor_flash_flush
adafruit#13 0x0004490e in filesystem_flush
adafruit#14 0x00043e18 in cleanup_after_vm
adafruit#15 0x0004414c in run_repl
adafruit#16 0x000441ce in main
```
When this happened -- which was inconsistent -- the display would keep
some heap allocation across reset which is exactly what we need to avoid.

NULLing the pointer in reconstruct follows what RGBMatrix does, and that
code is a bit more battle-tested anyway.

If I had a motivation for structuring the SharpMemory code differently,
I can no longer recall it.

Testing performed: Ran my complicated calculator program over multiple
iterations without observing signs of heap corruption.

Closes: adafruit#3473
jepler added a commit that referenced this pull request Oct 1, 2020
If `supervisor_start_terminal` is called twice in a row without
`supervisor_stop_terminal`, it would lose track of a supervisor
allocation.

This can occur when setting the rotation of a display, including the
way that a FramebufferDisplay sets rotation _AFTER_ initial construction,
first with a stack like
```
#0  supervisor_start_terminal
#1  in displayio_display_core_construct
#2  in common_hal_framebufferio_framebufferdisplay_construct
```
and then with a stack like
```
#0  supervisor_start_terminal
#1  in common_hal_framebufferio_framebufferdisplay_construct
#2  in framebufferio_framebufferdisplay_make_new
```
.. without an intervening stop_terminal call.

For reasons I didn't fully explore, this did not become a problem until
the ability to re-allocate a freed supervisor allocation was
implemented in adafruit#3482.  Demonstrating the problem requires adafruit#3482 + adafruit#3497
+ this PR.
tannewt pushed a commit that referenced this pull request Oct 1, 2020
jepler pushed a commit that referenced this pull request Nov 11, 2020
jepler pushed a commit that referenced this pull request Feb 10, 2021
Merge latest CircuitPython changes (6.0.0-rc.1)
jepler pushed a commit that referenced this pull request Mar 23, 2021
update to most recent master
jepler pushed a commit that referenced this pull request Apr 28, 2021
jepler pushed a commit that referenced this pull request Apr 28, 2021
Merge from original repo
jepler pushed a commit that referenced this pull request Apr 28, 2021
jepler added a commit that referenced this pull request May 10, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at
each of p and q, even for values of p and q that have a difference
earlier.  Accessing additional values is frequently done in practice,
reading 4 or more bytes from each input at a time for efficiency, so
when completing "non_exist<TAB>" in the repl, this causes a diagnostic:

```
==16938==ERROR: AddressSanitizer: global-buffer-overflow on address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fffffffd1d0
READ of size 9 at 0x555555cd8dc8 thread T0
    #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
    #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
    #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/readline.c:225
    #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
    #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/unix/main.c:194
    adafruit#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/main.c:673
    adafruit#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/main.c:436
    adafruit#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
    adafruit#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/unix/micropython-coverage+0x40bd69)

0x555555cd8dc8 is located 0 bytes to the right of global variable 'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of size 8
  'import_str' is ascii string 'import '
SUMMARY: AddressSanitizer: global-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
Shadow bytes around the buggy address:
  0x0aab2ab93160: 04 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93170: 05 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93180: 06 f9 f9 f9 f9 f9 f9 f9 06 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93190: 05 f9 f9 f9 f9 f9 f9 f9 00 00 f9 f9 f9 f9 f9 f9
  0x0aab2ab931a0: 00 00 00 00 00 00 00 00 04 f9 f9 f9 f9 f9 f9 f9
=>0x0aab2ab931b0: 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9 f9
  0x0aab2ab931c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2ab931d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2ab931e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 f9
  0x0aab2ab931f0: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 f9 f9 f9
  0x0aab2ab93200: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==16938==ABORTING
```
jepler added a commit that referenced this pull request May 10, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at
each of p and q, even for values of p and q that have a difference
earlier.  Accessing additional values is frequently done in practice,
reading 4 or more bytes from each input at a time for efficiency, so
when completing "non_exist<TAB>" in the repl, this causes a diagnostic:

```
==16938==ERROR: AddressSanitizer: global-buffer-overflow on address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fffffffd1d0
READ of size 9 at 0x555555cd8dc8 thread T0
    #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
    #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
    #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/readline.c:225
    #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
    #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/unix/main.c:194
    adafruit#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/main.c:673
    adafruit#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/main.c:436
    adafruit#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
    adafruit#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/unix/micropython-coverage+0x40bd69)

0x555555cd8dc8 is located 0 bytes to the right of global variable 'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of size 8
  'import_str' is ascii string 'import '
SUMMARY: AddressSanitizer: global-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
Shadow bytes around the buggy address:
  0x0aab2ab93160: 04 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93170: 05 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93180: 06 f9 f9 f9 f9 f9 f9 f9 06 f9 f9 f9 f9 f9 f9 f9
  0x0aab2ab93190: 05 f9 f9 f9 f9 f9 f9 f9 00 00 f9 f9 f9 f9 f9 f9
  0x0aab2ab931a0: 00 00 00 00 00 00 00 00 04 f9 f9 f9 f9 f9 f9 f9
=>0x0aab2ab931b0: 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9 f9
  0x0aab2ab931c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2ab931d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aab2ab931e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 f9
  0x0aab2ab931f0: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 f9 f9 f9
  0x0aab2ab93200: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==16938==ABORTING
```

Signed-off-by: Jeff Epler <jepler@gmail.com>
jepler added a commit that referenced this pull request May 10, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at
each of p and q, even for values of p and q that have a difference
earlier.  Accessing additional values is frequently done in practice,
reading 4 or more bytes from each input at a time for efficiency, so
when completing "non_exist<TAB>" in the repl, this causes a diagnostic:

```
==16938==ERROR: AddressSanitizer: global-buffer-overflow on
address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
READ of size 9 at 0x555555cd8dc8 thread T0
    #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
    #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
    #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
    #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
    #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
    adafruit#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
    adafruit#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
    adafruit#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
    adafruit#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni

0x555555cd8dc8 is located 0 bytes to the right of global variable
'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
size 8
  'import_str' is ascii string 'import '
```

Signed-off-by: Jeff Epler <jepler@gmail.com>
jepler added a commit that referenced this pull request May 10, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at
each of p and q, even for values of p and q that have a difference
earlier.  Accessing additional values is frequently done in practice,
reading 4 or more bytes from each input at a time for efficiency, so
when completing "non_exist<TAB>" in the repl, this causes a diagnostic:

```
==16938==ERROR: AddressSanitizer: global-buffer-overflow on
address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
READ of size 9 at 0x555555cd8dc8 thread T0
    #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
    #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
    #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
    #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
    #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
    adafruit#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
    adafruit#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
    adafruit#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
    adafruit#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni

0x555555cd8dc8 is located 0 bytes to the right of global variable
'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
size 8
  'import_str' is ascii string 'import '
```

Signed-off-by: Jeff Epler <jepler@gmail.com>
jepler added a commit that referenced this pull request Jun 8, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at each
of p and q, even for values of p and q that have a difference earlier.
Accessing additional values is frequently done in practice, reading 4 or
more bytes from each input at a time for efficiency, so when completing
"non_exist<TAB>" in the repl, this causes a diagnostic:

    ==16938==ERROR: AddressSanitizer: global-buffer-overflow on
    address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
    READ of size 9 at 0x555555cd8dc8 thread T0
        #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
        #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
        #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
        #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
        #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
        adafruit#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
        adafruit#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
        adafruit#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
        adafruit#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni

    0x555555cd8dc8 is located 0 bytes to the right of global variable
    'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
    size 8
      'import_str' is ascii string 'import '

Signed-off-by: Jeff Epler <jepler@gmail.com>
jepler pushed a commit that referenced this pull request May 14, 2022
jepler pushed a commit that referenced this pull request Oct 1, 2022
jepler pushed a commit that referenced this pull request Nov 30, 2022
Typo in docs for supervisor.ticks_ms()
jepler pushed a commit that referenced this pull request May 4, 2023
jepler pushed a commit that referenced this pull request May 24, 2023
Board files for LOLIN S3 MINI
jepler pushed a commit that referenced this pull request Oct 25, 2023
Add board def for MagiClick S3
jepler pushed a commit that referenced this pull request Feb 28, 2024
jepler pushed a commit that referenced this pull request Jul 23, 2024
Added additional pin names to pins.c for vidi_x board
jepler pushed a commit that referenced this pull request Mar 2, 2025
@jepler jepler deleted the appveyor-circuitpython branch March 27, 2025 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant