如何在Cython中正确管理C ++对象的生命周期？

When writing a Cython wrapper for a C++ library, I've encountered a case where it's not clear how to correctly decide when to delete certain C++ instances.

在为C ++库编写Cython包装器时,我遇到了一个不清楚如何正确决定何时删除某些C ++实例的情况。

The C++ library looks something like this:

C ++库看起来像这样:

#include <stdio.h>
#include <string.h>

class Widget {
    char *name;
    public:
        Widget() : name(strdup("a widget")) {}
        ~Widget() { printf("Widget destruct\n"); }
        void foo() { printf("Widget::foo %s\n", this->name); }
};

class Sprocket {
    private:
        Widget *important;

    public:
        Sprocket(Widget* important) : important(important) {}
        ~Sprocket() { important->foo(); }
};

An important aspect of this library is that the Sprocket destructor uses the Widget* it was given, so the Widget must not be destroyed until after the Sprocket has been.

这个库的一个重要方面是Sprocket析构函数使用它给出的Widget *,因此在Sprocket发布之前不能销毁Widget。

The Cython wrapper I've written looks like this:

我写的Cython包装器看起来像这样:

cdef extern from "somelib.h":
    cdef cppclass Widget:
        pass

    cdef cppclass Sprocket:
        Sprocket(Widget*)


cdef class PyWidget:
    cdef Widget *thisptr

    def __init__(self):
        self.thisptr = new Widget()

    def __dealloc__(self):
        print 'PyWidget dealloc'
        del self.thisptr


cdef class PySprocket:
    cdef PyWidget widget
    cdef Sprocket *thisptr

    def __init__(self, PyWidget widget):
        self.widget = widget
        self.thisptr = new Sprocket(self.widget.thisptr)


    def __dealloc__(self):
        print 'PySprocket dealloc with widget', self.widget
        del self.thisptr

After building the Python build like this:

在构建Python构建之后:

$ cython --cplus somelib.pyx 
$ g++ -I/usr/include/python2.6 -L/usr/lib somelib.cpp -shared -o somelib.so
$

In the trivial case, it appears to work:

在琐碎的情况下,它似乎工作:

$ python -c 'from somelib import PyWidget, PySprocket
spr = PySprocket(PyWidget())
del spr
'
PySprocket dealloc with widget <somelib.PyWidget object at 0xb7537080>
Widget::foo a widget
PyWidget dealloc
Widget destruct
$

The cdef Widget field keeps the PyWidget alive until after PySprocket.__dealloc__ destroys the Sprocket. However, as soon as the Python garbage collected gets involved, the tp_clear function Cython constructs for PySprocket messes this up:

cdef Widget字段使PyWidget保持活动状态,直到PySprocket .__ dealloc__销毁Sprocket。但是,一旦收集到Python垃圾,tp_clear函数Cython为PySprocket构造就会混淆:

$ python -c 'from somelib import PyWidget, PySprocket
class BadWidget(PyWidget):
    pass
widget = BadWidget()
sprocket = PySprocket(widget)
widget.cycle = sprocket
del widget
del sprocket
'
PyWidget dealloc
Widget destruct
PySprocket dealloc with widget None
Widget::foo ��h�

Since there's a reference cycle, the garbage collector invokes the tp_clear to try to break the cycle. Cython's tp_clear drops all references to Python objects. Only after this happens does PySprocket.__dealloc__ get to run.

由于存在引用循环,因此垃圾收集器调用tp_clear来尝试打破循环。 Cython的tp_clear删除了对Python对象的所有引用。只有在这种情况发生后才能运行PySprocket .__ dealloc__。

Cython documentation warns about __dealloc__ (although it took me a while to learn what conditions it was talking about, since it doesn't go into any detail). So perhaps this approach is entirely invalid.

Cython文档警告__dealloc__(尽管我花了一些时间来了解它所讨论的条件,因为它没有详细说明)。所以也许这种方法完全无效。

Can Cython support this use case?

Cython可以支持这个用例吗?

As (what I hope is) a temporary work-around, I've moved to an approach that looks something like this:

作为(我希望是)一个临时的解决方案,我已经采用了一种看起来像这样的方法:

cdef class PySprocket:
    cdef void *widget
    cdef Sprocket *thisptr

    def __init__(self, PyWidget widget):
        Py_INCREF(widget)
        self.widget = <void*>widget
        self.thisptr = new Sprocket(self.widget.thisptr)


    def __dealloc__(self):
        del self.thisptr
        Py_DECREF(<object>self.widget)

In other words, hiding the reference from Cython so that it is still valid in __dealloc__, and doing reference counting on it manually.

换句话说,隐藏Cython中的引用,使其在__dealloc__中仍然有效,并手动对其进行引用计数。

1 个解决方案

#1

cdef extern from "somelib.h":
    cdef cppclass Widget:
        pass

    cdef cppclass Sprocket:
        Sprocket(Widget*)


cdef class PyWidget:
    cdef Widget *thisptr
    cdef set    sprockets

    def __init__(self):
        self.thisptr = new Widget()
        self.sprockets = set()

    def __dealloc__(self):
        print 'PyWidget dealloc'
        #PyWidget knows the sprockets and notifies them on destroy
        sprockets_to_dealloc = self.sprockets.copy()
        #with this solution spr items can call back to detach
        for spr in sprockets_to_dealloc:
          del spr
        del self.thisptr

    def attach(PySprocket spr):
        print 'PySprocket attach'
        self.sprockets.add(spr)

    def detach(PySprocket spr):
        print 'PySprocket detach'
        self.sprockets.remove(spr)

cdef class PySprocket:
    cdef PyWidget widget
    cdef Sprocket *thisptr

    def __init__(self, PyWidget widget):
        self.thisptr = new Sprocket(widget.thisptr)
        #You should be sure here that the widget exists
        widget.attach(self)
        self.widget = widget

    def __dealloc__(self):
        self.widget.detach(self)
        del self.thisptr

---

I come back a bit later to check what I have wrote, cause I'm quite tired, but here is what matters: The point is that you want to notify Sprockets when destroying Widget, and vice versa.

我稍后回来查看我写的内容,因为我很累,但重要的是:重点是你要在销毁Widget时通知Sprockets,反之亦然。

It is a general solution, can be tuned up.

这是一个通用的解决方案,可以调整。

You have to include error handling also, I have skipped that absolutely. Nothing to do with garbage collector, there was a design problem in your code.

你还必须包括错误处理,我绝对跳过了。与垃圾收集器无关,代码中存在设计问题。

EDIT: these codes are equialent:
A

编辑:这些代码是等价的:A

class BadWidget(PyWidget):
    pass
widget = BadWidget()
sprocket = PySprocket(widget)
widget.cycle = sprocket ###1
del widget ###2
del sprocket

B

class BadWidget(PyWidget):
    pass
widget = BadWidget()
sprocket = PySprocket(widget)
sprocket.widget.cycle = sprocket ###1
del sprocket.widget ###2
del sprocket

###2 will call sprocket.widget.__deallocate__() and it doesn't deallocates sprocket.widget.cycle, so the sprocket will survive the widget

### 2将调用sprocket.widget .__ deallocate __()并且它不会释放sprocket.widget.cycle,因此sprocket将在小部件中存活