Skip to content

gc_realloc returns NULL after a few calls #322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iabdalkader opened this issue Feb 25, 2014 · 9 comments
Closed

gc_realloc returns NULL after a few calls #322

iabdalkader opened this issue Feb 25, 2014 · 9 comments

Comments

@iabdalkader
Copy link
Contributor

Hi, I think there's a bug in gc_realloc, after calling it a few times it returns NULL even though there's enough memory to satisfy the request.. This is the code I'm using to test:

void test_realloc()
{
    printf("testing realloc\n");
    void *mem = gc_alloc(100);
    for (int i=1; i<=50; i++) { /* ~50K */
        mem = gc_realloc(mem, 1000*i);
        printf("realloc: %d\n", 1000*i);
        if (mem==NULL) {
            pyb_info();
            printf("\r\n");
            sys_tick_delay_ms(5000);
            __asm__ volatile ("BKPT");
        }
    }

}

I call test_realloc() after running the main script in main.c

This is the result:

realloc: 37000 <<< stops here
ID=29003600:0a473133:32353735
S=168000000
H=168000000
P1=42000000
P2=84000000
_text_end=8043b50
_data_start_init=8043b50
_data_start=20000000
_data_end=200001fc
_bss_start=20000200
_bss_end=20001f78
_stack_end=20020000
_ram_start=20000000
_heap_start=20001f78
_heap_end=2001c000
_ram_end=20020000
qstr:
  n_pool=1
  n_qstr=17
  n_str_data_bytes=213
  n_total_bytes=309
GC:
  104960 total
used:  37888  free:67072 <<--- 67K remaining 
  1=52 2=2 m=2250
LFS free: 30720 bytes

Code is compiled with optimization and tested on discovery board... Can you confirm this ?

@dpgeorge
Copy link
Member

It could be that the 37k block is allocated right in the middle of the heap, and so no contiguous 38k block exists. I've added a function to the GC to print out the organisation of the heap and what is allocated/free. Call gc_dump_alloc_table after pyb_info to see it. It'll be large, but will tell you exactly what is going on.

@dpgeorge
Copy link
Member

Note that, for your test case above, the GC can be improved to make it work. At the moment a realloc is just an alloc followed by a memcpy followed by a free. This could be changed to see if the current block can be extended in place. See line 334 of gc.c.

@iabdalkader
Copy link
Contributor Author

but this is called first thing after running main.py, and nothing else is allocated, or maybe small objects for the strings printed are allocated which cause fragmentation :? anyway, I will try the function and send back the results.

@dpgeorge
Copy link
Member

But you are allocating 1k, then 2k, then 3k, ..., 36k before allocating the 37k, so this will fragment the memory quite a bit.

@iabdalkader
Copy link
Contributor Author

what if I free the memory before I look for a bigger slice, would that work :) ? oh right, need to copy it nevermind

@dpgeorge
Copy link
Member

Yes, if you changed realloc to free+alloc, then it should work....

@iabdalkader
Copy link
Contributor Author

I saw the comment you mentioned in gc.c I want to give this a try, so let's leave this issue open for now...

@dpgeorge
Copy link
Member

It should be easy to implement proper realloc: code from gc_nbytes can help determine if there are enough blocks after the current one to satisfy the allocation, and the last few lines of gc_alloc can be used to mark these blocks as used (ATB_FREE_TO_TAIL).

iabdalkader added a commit to openmv/micropython that referenced this issue Mar 5, 2014
@iabdalkader
Copy link
Contributor Author

I tried the same test again, and it didn't fail this time:

void test_realloc()
{
    while (!usb_vcp_is_connected());
    printf("\n\ntesting realloc\n");
    void *mem = gc_alloc(1024);
    for (int i=1; i<=50; i++) { /* ~50K */
        mem = gc_realloc(mem, 1024*i);
        printf("realloc: %p %d\n\n", mem, 1024*i);
        if (mem==NULL) {
            pyb_info();
            while(1);
        }
    }
    pyb_info();

    mem = gc_realloc(mem, 1024);
    pyb_info();
    while(1);
}

result after first calls to realloc:

  102528 total
used: 52944 free: 49584
  1=40 2=5 m=3200
LFS free: 95744 bytes

result after realloc'ing down to 1024

GC:
  102528 total
used: 2768 free: 99760
  1=40 2=5 m=64
LFS free: 95744 bytes

@dpgeorge dpgeorge closed this as completed Mar 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants