The reason for this seems to be the implementation of BaseException's reduce:

/* Pickling support */
static PyObject *
BaseException_reduce(PyBaseExceptionObject *self)
{
    if (self->args && self->dict)
        return PyTuple_Pack(3, Py_TYPE(self), self->args, self->dict);
    else
        return PyTuple_Pack(2, Py_TYPE(self), self->args);
}

In our hacked implementation that injects BaseException into the tp_bases of .NET exceptions, args and dict are both NULL, so the Decref on the result of this function will fail, could be that even the cast to PyBaseExceptionObject is already broken.

Can't you use dill instead pickle?

I just tried and it works!

In [1]: import clr
In [2]: import dill
In [3]: dill.dumps(Exception())
Out[3]: '\x80\x02cexceptions\nException\nq\x00)Rq\x01.'

On Mon, Nov 7, 2016, 11:00 AM Benedikt Reinartz notifications@github.com
wrote:

The reason for this seems to be the implementation of BaseException's reduce
:

/* Pickling support */
static PyObject *
BaseException_reduce(PyBaseExceptionObject *self)
{
if (self->args && self->dict)
return PyTuple_Pack(3, Py_TYPE(self), self->args, self->dict);
else
return PyTuple_Pack(2, Py_TYPE(self), self->args);
}

In our hacked implementation that injects BaseException into the tp_bases
of .NET exceptions, args and dict are both NULL, so the Decref on the
result of this function will fail, could be that even the cast to
PyBaseExceptionObject is already broken.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#284 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5WhAoHLvz5WLP6LHqpk84jwvgP4zks5q71lBgaJpZM4KrQec
.

I could maybe do that to silence this particular error on my end, yes. However, this doesn't really fix the problem that you can easily kill the interpreter using this ;)

It happened to me when returning an exception object from a function run in a ProcessPoolExecutor, very difficult to debug ...

I agree that crashing interpreter is not good :)

If dill does not work for your case, then there is also cloudpickle used by
PySpark.

On Mon, Nov 7, 2016 at 11:11 AM, Benedikt Reinartz <notifications@github.com

wrote:

I could maybe do that to silence this particular error on my end, yes.
However, this doesn't really fix the problem that you can easily kill the
interpreter using this ;)

It happened to me when returning an exception object from a function run
in a ProcessPoolExecutor, very difficult to debug ...

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#284 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5fO8ShvOtXHpA-Z6-APAyPfSVqhVks5q71upgaJpZM4KrQec
.

this also works with cPickle (pickle speed up with C), but gives a different hash:

In [3]: dill.dumps(Exception())
Out[3]: '\x80\x02cexceptions\nException\nq\x00)Rq\x01.'
In [4]: import cloudpickle
In [5]: cloudpickle.dumps(Exception())
Out[5]: '\x80\x02cexceptions\nException\nq\x00)Rq\x01.'
In [6]: import cPickle
In [7]: cPickle.dumps(Exception())
Out[7]: 'cexceptions\nException\np1\n(tRp2\n.'

2 comments:

1. You are apparently using Python 2 (cPickle), which uses an additional hack to make the exceptions work as old-style classes. I'm using Python 3.
2. Are you sure you are using a System.Exception? In anycase, the output looks like it will just unpickle to an exceptions.Exception instance.