You are correct that it does not work! In fact, the special method __del__ is not implemented for user-defined classes.

I know that a good programming practice is to assume that __del__ may never be called, but hey, this still should work :)

Where did you got that from, i'm curious ... i guess you read too much javascript XD
if __del__ isn't called that means either gc is not collecting, or that a ref is still holding.

@dpgeorge, This should be implemented and active on unix port. Because when you load C++ classes via ffi wrappers ( ie from Python not from a C module ) you need to call dtor explicitely.

use case: MicroPython uPanda3D bindings (WIP, desktop and web )

eg https://github.com/pmp-p/panda3d-interrogator/blob/master/tests/testpy.py#L149 should not have to be called manually

will MICROPY_ENABLE_FINALISER fix that ?

Where did you got that from, i'm curious ... i guess you read too much javascript XD
if del isn't called that means either gc is not collecting, or that a ref is still holding.

cPython docs: data model Well worth reading the section on __del__.

tl;dr

It is not guaranteed that del() methods are called for objects that still exist when the interpreter exits.

Yes, same should be true for MicroPython: GC doesn't run on interpreter exit as far as I know, so if your object didn't get GC'd during running it __del__ just doesn't get called.

will MICROPY_ENABLE_FINALISER fix that ?

That adds support for calling the finaliser when an object gets garbage collected. So the aforementioned problem still exists.

It is not guaranteed that del() methods are called for objects that still exist when the interpreter exits.

for objects that still exist , and why should there any left apart from circular references which are corners case in most gc and programmer responsability, i'm still curious ...

@stinos you made the point, that's because someone did forgot to cleanup on exit that __del__ would not be called. Otherwise finalizers should always be called for normal operations.

for objects that still exist

Does this not just mean:

a = 35
# Now when this script exits, a still exists

?

i would not now i rarely use globals(), and i usually call del if don't want them to show up in inspection mode. on that snippet it's obvious you want a to survive interactive loop exit ( interpreter does not always exits there see "-i" invocation )

So i did put some C++ classes in locals() ( as in a function ) , run that function , call del on function, gc.collect() and ... __del__ was never called. I hope it's only lack of coffee.

I'm not sure del func implies 'get rid of every variable created by func', and I'm also not sure the GC is that deterministic.

I'm also not sure the GC is that deterministic.

At first glance, I would agree. And it's a bit annoying.

reading #245
__del__ is not implemented on Python user defined classes at all but no need to reopen an issue as this one seems appropriate.

@dpgeorge @stinos c++ ffi handling needs python finalizers there's absolutely no need to force using precompiled C modules.

wip use case: https://github.com/pmp-p/panda3d-interrogator/blob/master/build/upanda3d.py
( generic c++ loader https://github.com/pmp-p/panda3d-interrogator/blob/master/interrogator/uplusplus.py )

One more thought since this hasn't been mentioned explicitly: __del__ is not the exact counterpart of a destructor in C++ and assuming it is or relying on it for destruction is going to lead to surprises, i.e. usually undefined behavior, at one point or another. So always take that into account when you do use it, and if you really want something deterministic use a context manager instead.

in my use case __del__ is used to synchronize gc beetween Python and underlying (dead) C++ objects, nothing is done in destructors other than reclaiming memory ( Panda3D is an engine designed with cpython+refcounting in mind). On the other hand C++ is as good as any for writing extensions.

Of course i can use a context manager and even record state of global/locals and do mark and sweep on exit and also solve circular references ... but why ? better just do what __del__ should do.

ie thank you for not turning MicroPython in an obsolete javascript implementation ( FYI modern js is getting weakref and finalizers )

@stinos

So always take that into account when you do use it

Are you specifically talking to me ? and what's the link with garbage collection ?

if you really want something deterministic use a context manager instead.

that's the sure way to crash with untracked (circular or not) references created within the context ( cf javascript ). so same as above thank you.

eg:

with cpp_instance() as o :
    untracked=o

untracked.crash()
o.crash() # context manager is definitively not "scoping" !

@dpgeorge i'd really would like to know about __del__ support and eventually a timeline for it, since my C++ generic classes loader is almost complete and ffi support is in progress of being fixed on emscripten.

I don't want to spend time generating plain C modules for interfacing C++ if not needed as i would not know how to use https://github.com/stinos/micropython-wrap for that.

what's the link with garbage collection

That even if you implement a destructor of a C++ object in __del__, the Python scoping rules are different from the C++ ones, meaning for instance that a variable going out of scope in Python does not automatically leads to __del__ getting called but instead that gets triggered by garbage collection.

that's the sure way to crash with untracked (circular or not) references created within the context ( cf javascript )

My point was rather that using a context manager will deterministically allow you to clean up resources in __exit__ (i.e. it's deterministic whereas __del__ is not). But that's orthogonal to the problem you are describing, which is just the effect of trying to use Python like it's C++.

i would not know how to use https://github.com/stinos/micropython-wrap for that

Can you elaborate what exactly isn't clear, I'd be happy to clarify the Readme (but maybe better raise an issue in the repository itself in order not to derail this thread too much).

I was curious what would be needed to implement this and unless I'm missing something it's a rather small change (with a performance impact though). In mp_obj_new_instance in objtype.c change

mp_obj_instance_t *o = m_new_obj_var(mp_obj_instance_t, mp_obj_t, num_native_bases);

to

mp_obj_instance_t *o = m_new_obj_var_with_finaliser(mp_obj_instance_t, mp_obj_t, num_native_bases);

and garbage collecting of user-defined classes will result in __del__ getting called if present from a simple test cas I tried. YMMV..

@dpgeorge you know if this is really all that's needed? If so this seems worth implementing conditionally.

meaning for instance that a variable going out of scope in Python does not automatically leads to del getting called but instead that gets triggered by garbage collection.

in that case scoping is of litle interest, only the finalizer getting called now or later or maybe never matters, but only gc can trigger that ( given circular ref can be solved ).

mp_obj_instance_t *o = m_new_obj_var_with_finaliser(mp_obj_instance_t, mp_obj_t, num_native_bases);

i'd prefer first a Python way to do that since MicroPython primary use is interpreting Python code maybe with the help of one specialized object type with native finalizer and callback handler:

# cpython demonstration of a what could be a µpy C "uweak" object
class uweak:
    def __init__(self, cb, key):
        self.key = key
        self.cb = cb

    def __del__(self):
        self.cb( self.key )
 #============================

def dtor(the_key_refing_a_pointer):
      #ffi magic  dealing with key to dtor the pointer
     print("cleaned up", the_key_refing_a_pointer )

class user_def:
    def __init__(self ):
        self.weak =uweak(dtor, id(self))

def test():
    u = user_def()
    # u vanishes here

test()
#collect happens -> dtor called

ref : https://github.com/tc39/proposal-weakrefs

@stinos

Can you elaborate what exactly isn't clear, I'd be happy to clarify the Readme (but maybe better raise an issue in the repository itself in order not to derail this thread too much).

in case of FFI not working on emscripten or missing finalizer on MP the only way for me to complete my project ( a standalone wasm/wasi Panda3D runtime ) would be to build a C/C++ MicroPython module boxing C++ pointers , methods calls , enums , return types conversion to Python objects : just to get __del__ working. i'm pretty sure your project does parts of that list if not all , but i don't code in C++ so i have no idea how to convert the logic of https://github.com/pmp-p/panda3d-interrogator/blob/master/interrogator/uplusplus.py to deal with extern C++ objects ( i can still provide a C interface if needed ).

i'd prefer first a Python way to do that

Not possible I think, there's no place where the core C code taking care of finalizers gets exposed to Python now.

i'm pretty sure your project does parts of that list if not all

Most of it I think yes but I don't yet have something for converting enums automaitcally for instance, and you do have to write the registration for each class/function, see the module.cpp file in the repository. Btw I don't think we ever had any problems using circular references or saw the need for weakref, which might be because we just wrap everything in a shared_ptr because that roughly matches with how Python deals with objects (and yes, __del__ calls the shared_ptr's destructor).

in continuation of cross posting #5106 and #4993 as ref

ANY kind of user finalizers ( personnally i'm in favour of JS group finalizers kind ).

Not sure what you mean exactly, but if anything gets implemented, for such a core feature we cannot really deviate from the standard imo and there's only one type of finalizer?

then define a RFC for a group finalizer and a MEP ie create a standard ?
that's valid because many Python VM don't implement __del__ and they have sometimes good reason (brython) or blocker ( jython ).

my proposal is :
1 builtin object type that is tracked by GC not every object created !

If people need __del__ everywhere they just have to subclass and override builtin.object

The people who would need that are probably experienced anyway, so just give them 1 tool ( cheap of course but filling the void ).

my code is clear because it's a working POC.
i don't think gc type, nan-boxing, marksweep/moving/refcounting, whatever have a real influence on final product provided to Python and simplicity of the tool.
So it should not block anything or anyone. __del__ was (is) bad but lack of any finalizer is worse.

Removing things to see what's really needed is a valid approach, but not moving even when javascript show you the way to do it (finally!) at low cost is not.

The C line pasted mp_obj_instance_t *o = m_new_obj_var_with_finaliser(mp_obj_instance_t, mp_obj_t, num_native_bases); is less clear, and does not make any sense to me on Python side without pointer to doc/example/explanation. ( if i was about to make a PR which i was not asked for and have no particular interest in doing just now ).

without pointer to doc/example/explanation

Currently the only documentation and example is the code itself (which is fairly readable, given some C or similar background) and references to it here and on the forum..

The explanation for this one is fairly straightforward: m_new_obj_var_with_finaliser tells the allocator to create a new object and set a bit somewhere so when the object gets finalized, it gets scanned to see if it has __del__ and if so that gets called. Objects not allocated like that won't have that bit set. Only types defined in C can choose whether they use the function or not. So if an object of a type defined in Python wants that functionality, it either needs to be allocated with said function, or following your proposal for instance the core needs a new function which allows setting that 'this object has a finaliser' flag ad hoc and that function should be made available to Python which would enable manually marking one or more specific types del'able.

the core needs a new function which allows setting that 'this object has a finaliser' flag

That's i was missing : i thought having a del method on C object in a C module with a python instanciation would be autodetected just because finalizer are enabled.

So if i get it right the C python "normal" object could create and hold a child with m_new_obj_var_with_finaliser without requiring anything out of the module ? that seems enough for a group finalizer.

thx !

Sidenote i don't have much time for forum, i try to add bits to MP when i'm idle/compiling my primary target for my projects is still cpython on mobile/web so the lag in other subjects is not helping to crunch efficiently that time.

https://forum.micropython.org/search.php?keywords=new_obj_var_with_finaliser => 0
https://forum.micropython.org/viewtopic.php?f=3&t=3290&p=39328&hilit=__del__#p39328 => 0 move.

So if i get it right the C python "normal" object could create and hold a child with m_new_obj_var_with_finaliser

Something like that yes. There's more than one way to achieve your proposal, apart from the example I gave with setting the flag one could also implement a type in C which takes a callback to allow calling an arbitrary function in it's finaliser, or have a C type which just has finaliser support and is meant to be subclassed in Python, or ...

(note I edited my previous comment because I wrote it without checking the code, and it stated all C types use a finaliser by default but that's not true)

For reference, here's an implementation of a type which calls an arbitrary function in it's finalizer

typedef struct _cb_finalizer_t {
    mp_obj_base_t base;
    mp_obj_t fun;
} cb_finalizer_t;

extern const mp_obj_type_t mp_type_cb_finalizer;

STATIC mp_obj_t new_cb_finalizer(const mp_obj_type_t *type_in, size_t n_args, size_t n_kw, const mp_obj_t *args) {
    mp_arg_check_num(n_args, n_kw, 1, 1, false);
    cb_finalizer_t *o = m_new_obj_with_finaliser(cb_finalizer_t);
    o->base.type = &mp_type_cb_finalizer;
    o->fun = args[0];
    return o;
}

STATIC mp_obj_t cb_finalizer_del(mp_obj_t self_in) {
    return mp_call_function_0(((cb_finalizer_t *)self_in)->fun);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_1(cb_finalizer_del_obj, cb_finalizer_del);

STATIC void cb_finalizer_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
    if (dest[0] == MP_OBJ_NULL && attr == MP_QSTR___del__) {
        dest[0] = MP_OBJ_FROM_PTR(&cb_finalizer_del_obj);
        dest[1] = self_in;
    }
}

const mp_obj_type_t mp_type_cb_finalizer = {
    {&mp_type_type},
    .name = MP_QSTR_cb_finalizer,
    .make_new = new_cb_finalizer,
    .attr = cb_finalizer_attr,
};

After adding this to modgc.c i.e. add { MP_ROM_QSTR(MP_QSTR_cb_finalizer), MP_ROM_PTR(&mp_type_cb_finalizer) }, to mp_module_gc_globals_table, this example will print i am a finalizer:

import gc

# gc isn't entirely deterministic and without these lines
# the linux build doesn't always collect x
a = 1
b = 2
c = 3

def allocate_some():
  d = [a] * 100  # noqa
  e = [3] * 100  # noqa
  f = [4] * 100  # noqa

def foo():
  print('i am a finalizer')

def fun():
  x = gc.cb_finalizer(foo)  # noqa

fun()
allocate_some()

gc.collect()

great, that looks exactly what i have in mind for cleaning up c/c++ pointer list.
i'll test that ASAP. Thanks again !

refing for the doc completion v923z/micropython-usermod#1

Just curious if there has been progress or update on this?

I stumbled on it while trying to debug some code on an ESP32 running micropython 1.24.1.
(sysname='esp32', nodename='esp32', release='1.24.1', version='v1.24.1 on 2024-11-29', machine='Generic ESP32 module with ESP32')

It took me some time with Google to figure out what was going on and then to locate the small mention in the documentation which led me to this issue.

My biggest concern is that deleting an object & then calling gc.collect() will leave everything in the destroyed instance still in memory & still running (the class I've been working on uses a hardware timer & my del() shuts it down, although when I tried to destroy the object discovered that the timer was still running & still referencing variables which were part of the now destroyed object).

If you need to make objects that free/stop/cleanup resources when no longer used, the best way to do it is with a context manager. This will ensure that things are cleaned up, even if an exception occurs. The second best way is to make a close() method and ensure that close() is called when the object is no longer needed.

__del__ is non-deterministic by definition, so is not really the right tool for that job. See https://github.com/orgs/micropython/discussions/14287 as an example of how __del__ can be problematic in this way.