Skip to content

memory leak with math.log/exp #522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wrobell opened this issue Apr 25, 2014 · 10 comments
Closed

memory leak with math.log/exp #522

wrobell opened this issue Apr 25, 2014 · 10 comments

Comments

@wrobell
Copy link

wrobell commented Apr 25, 2014

When running

import math

def calc(data):
    #data = tuple(math.exp(1) for v in data)
    data = [math.log(1) for v in data]
    return data

data = list(range(16))
for i in range(10000):
    data = calc(data)

print(data)

I am getting

MemoryError:

Is that to be expected with current implementation of GC?
@pfalcon
Copy link
Contributor

pfalcon commented Apr 25, 2014

Please describe how exactly you run it, and see #510.

@dpgeorge
Copy link
Member

I also get MemoryError. Even replacing math.log(1) with 0.0 gives MemoryError. I would say that it should not be running out of RAM so easily.

@pfalcon
Copy link
Contributor

pfalcon commented Apr 25, 2014

And I get [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

@dpgeorge
Copy link
Member

The following also gives MemoryError:

def calc(data):
    data = [0.0 for v in range(16)]

data = list(range(16))
for i in range(10000):
    data = calc(data)

print(data)

gcc-4.8.2, 64-bit.

@pfalcon
Copy link
Contributor

pfalcon commented Apr 25, 2014

Welcome to what I usually experienced with 4.8 on 32bit ;-). May I now push following?

 void *m_malloc_fail(int num_bytes) {
     DEBUG_printf("memory allocation failed, allocating %d bytes\n", num_bytes);
+    printf("memory allocation failed, allocating %d bytes\n", num_bytes);
     nlr_raise((mp_obj_t)&mp_const_MemoryError_obj);
 }

Because I got too tired to stash/unstash it. When you apply it, you'll likely see that some nonsencial alloc request is coming in. It's actually random though, another phase of the moon you get nonsencial method called on str or int, another - segfault.

@dpgeorge
Copy link
Member

But we need to keep m_malloc_fail not printing anything out for "production" code, since it is legit to return a MemoryError in some cases (eg tests/memoryerror.py). And what about on pyboard?

@pfalcon
Copy link
Contributor

pfalcon commented Apr 25, 2014

That's why I didn't propose to enable it by default so far. ;-) So, let's decide if we're in "production" mode, or we've reached crux of writing a serious project - when we hit a hard problem, which lies on critical path to further progress.

@dpgeorge
Copy link
Member

Well okay, i'll just have to remember not to flash the pyboards with this version :)

@pfalcon
Copy link
Contributor

pfalcon commented Apr 25, 2014

Well, nope then, I'll better stash/unstash ;-). But let's have plan how to address this stuff (in #510).

dpgeorge added a commit that referenced this issue Apr 25, 2014
@dpgeorge
py, gc: Zero out newly allocated blocks in the GC.
daab651 
Also add some more debugging output to gc_dump_alloc_table().

Now that newly allocated heap is always zero'd, maybe we just make this
a policy for the uPy API to keep it simple (ie any new implementation of
memory allocation must zero all allocations).  This follows the D
language philosophy.

Before this patch, a previously used memory block which had pointers in
it may still retain those pointers if the new user of that block does
not actually use the entire block.  Eg, if I want 5 blocks worth of
heap, I actually get 8 (round up to nearest 4).  Then I never use the
last 3, so they keep their old values, which may be pointers pointing to
the heap, hence preventing GC.

In rare (or maybe not that rare) cases, this leads to long, unintentional
"linked lists" within the GC'd heap, filling it up completely.  It's
pretty rare, because you have to reuse exactly that memory which is part
of this "linked list", and reuse it in just the right way.

This should fix issue #522, and might have something to do with
issue #510.
@dpgeorge
Copy link
Member

Hopefully fixed. See daab651 and read commit message for details.

Thanks @wrobell for finding a simple case where this happened!

tannewt added a commit to tannewt/circuitpython that referenced this issue Apr 12, 2018
@tannewt
Add audio output support!
28642ab 
This evolves the API from 2.x (and breaks it). Playback devices are now
separate from the samples themselves. This allows for greater playback
flexibility. Two sample sources are audioio.RawSample and audioio.WaveFile.
They can both be mono or stereo. They can be output to audioio.AudioOut or
audiobusio.I2SOut.

Internally, the dma tracking has changed from a TC counting block transfers
to an interrupt generated by the block event sent to the EVSYS. This reduces
the overhead of each DMA transfer so multiple can occure without using up TCs.

Fixes micropython#652. Fixes micropython#522. Huge progress on micropython#263
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wrobell @pfalcon @dpgeorge