使用pyx/pxd文件中的函数工厂为c库生成cython函数包装器。

I am re-evaluating different ways to wrap external C libraries into Python. I had chosen long ago to use the plain Python C API, which was fast, simple, standalone and, as I thought, future-proof. Then I stumbled upon PyPy, which apparently does not plan on supporting the CPython API, but might become an interesting alternative in the future... I am thus looking for a higher-level entry point. ctypes was slow, so now I am back at cython, which appears to make an effort to support PyPy.

我正在重新评估用不同的方法将外部C库封装到Python中。我很早以前就选择了使用普通的Python C API，它快速、简单、独立，而且正如我所想的那样，是未来的证明。然后我偶然发现PyPy，它显然不打算支持CPython API，但是将来可能会成为一个有趣的替代方案……因此，我正在寻找一个更高层次的入口点。ctypes是很慢的，所以现在我又回到了cython，这似乎是为了支持PyPy。

My library has lots of functions with the same signature, so I made extensive use of C preprocessor macros to generate the Python module. I thought this would become a lot more comfortable in cython, since I would have access to the whole Python language. However, I am having trouble writing a factory for my function wrappers:

我的库具有很多具有相同签名的函数，因此我广泛使用C预处理器宏来生成Python模块。我认为这在cython中会更加舒适，因为我可以访问整个Python语言。然而，我在为我的函数包装设计工厂时遇到了麻烦:

import cython
from numpy cimport ndarray, double_t

cimport my_c_library

cdef my_c_library.data D

ctypedef double_t DTYPE_t

cdef parse_args(ndarray[DTYPE_t] P, ndarray[DTYPE_t] x, ndarray[DTYPE_t] y):
    D.n = P.size
    D.m = x.size
    D.P = <double*> P.data
    D.x = <double*> x.data
    D.y = <double*> y.data

def _fun_factory(name):
    cpdef fun(ndarray[DTYPE_t] P, ndarray[DTYPE_t] x, ndarray[DTYPE_t] y):
        parse_args(P, x, y)
        getattr(my_c_library, name)(&D)
        return y
    return fun

fun1 = _fun_factory('fun1')
fun2 = _fun_factory('fun2')
# ... many more function definitions ...

The cython compiler complains: "C function definition not allowed here", referring to the cpdef inside _fun_factory. What is the problem here? I thought pyx files were just like regular python files. Is there a way to get this working, other than the obvious to generate the pyx file dynamically from a separate python script, such as setup.py?

cython编译器抱怨:“此处不允许使用C函数定义”，指的是在_fun_factory内的cpdef。这里有什么问题?我认为pyx文件就像普通的python文件一样。除了从单独的python脚本(如setup.py)动态生成pyx文件之外，是否还有其他方法可以实现这个工作?

I was also surprised that cython wouldn't let me do:

我也很惊讶，cython不让我这么做:

ctypedef ndarray[double_t, ndim=1] p_t

to clean up the code. Why doesn't this work?

清理代码。为什么不工作呢?

I am aware that there are automatic C -> cython translators out there, but I am reluctant to make myself dependent on such 3rd party tools. But please feel free to suggest one if you think it is ready for production use.

我知道有自动的C ->的cython翻译，但是我不愿意让自己依赖于这样的第三方工具。但是，如果你认为它已经准备好用于生产，请随时提出建议。

2 个解决方案

#1

pyx files are not like Python files in the sense that you can match C and Python functions, and there are some constraints on what you can do with a C (cdef or cpdef) function. For one, you can't dynamically generate C code at runtime, which is what your code is trying to do. Since fun is really just executing some Python code after typechecking its arguments, you might just as well make it a regular Python function:

pyx文件与Python文件不一样，因为您可以匹配C和Python函数，并且在使用C (cdef或cpdef)函数时也有一些限制。首先，您不能在运行时动态生成C代码，这是您的代码正在尝试做的事情。由于fun实际上只是在对其参数进行类型检查之后执行一些Python代码，所以您可以将它作为一个正常的Python函数:

def fun(P, x, y):
    parse_args(P, x, y)
    getattr(my_c_library, name)(&D)
    return y

parse_args will do the same argument checking, so you lose nothing. (I'm not sure whether getattr works on a C library that's cimport'd, though. You might want to import it as well.)

parse_args将执行相同的参数检查，因此不会丢失任何东西。(我不确定getattr是否在cimport的C库中工作)。您可能也想导入它。

As for the ctypedef, that's probably some limitation/bug in Cython that no-one's got round to fixing yet.

对于ctypedef，这可能是在Cython中存在的一些限制/bug，没有人会去修复它。

#2

After playing around some more, the following appears to work:

在玩了更多的游戏之后，接下来的工作似乎是:

def _fun_factory(fun_wrap):
    def fun(P, x, y):
        parse_args(P, x, y)
        fun_wrap()
        return y
    return fun

def _fun1(): my_c_library.fun1(&D)
def _fun2(): my_c_library.fun2(&D)
# ... many more ...

fun1 = _fun_factory(_fun1)
fun2 = _fun_factory(_fun2)
# ... many more...

So there seems to be no possibility to use any Python operations on expressions like my_c_library.fun1(&D), which apparently need to be typed as is. The factory can be used only on a second pass when one already has generated a first set of Python wrappers. This isn't any more elegant than the obvious:

因此，似乎不可能在my_c_library.fun1(&D)这样的表达式上使用任何Python操作，这显然需要输入。当您已经生成了第一组Python包装器时，工厂只能在第二个通道上使用。这并不比显而易见的更优雅:

cpdef fun1(ndarray[DTYPE_t] P, ndarray[DTYPE_t] x, ndarray[DTYPE_t] y):
    parse_args(P, x, y)
    my_c_function.fun1(&D)
    return y

# ... many more ...

Here, cpdef can be used without problems. So I'm going for the copy-paste approach... Anyone also interested in preprocessor macros for Cython in the future?

这里，cpdef可以毫无问题地使用。所以我要用复制粘贴方法…对于未来的Cython，还有谁对预处理器宏感兴趣吗?