如何在Cython中使用128位整数

On my 64 bit computer the long long type has 64 bits.

在我的64位计算机上,long long类型有64位。

print(sizeof(long long))
# prints 8

I need to use 128 bit integers and luckily GCC supports these. How can I use these within Cython?

我需要使用128位整数,幸运的是GCC支持这些。我怎样才能在Cython中使用它们?

The following doesn't work. Compiling foo.pyx containing just

以下不起作用。编译foo.pyx只包含

cdef __int128_t x = 0

yields

$ cython foo.pyx 

Error compiling Cython file:
------------------------------------------------------------
...

cdef __int128_t x = 0
    ^
------------------------------------------------------------

foo.pyx:2:5: '__int128_t' is not a type identifier

3 个解决方案

#1

EDIT: this is NOT a workaround anymore, this is the right way to do it. Refer also to @IanH's answer.

编辑:这不再是一种解决方法,这是正确的方法。另请参阅@ IanH的答案。

Now, the problem you have is that cython does not recognize your type, while gcc does. So we can try to trick cython.

现在,你遇到的问题是cython无法识别你的类型,而gcc就是这样。所以我们可以尝试欺骗cython。

File helloworld.pyx:

cdef extern from "header_int128.h":
    # this is WRONG, as this would be a int64. it is here
    # just to let cython pass the first step, which is generating
    # the .c file.
    ctypedef unsigned long long int128

print "hello world"

cpdef int foo():
    cdef int128 foo = 4
    return 32

File header_int128.h:

typedef __int128_t int128;

File setup.py:

from distutils.core import setup
from Cython.Build import cythonize

setup(ext_modules = cythonize("helloworld.pyx"))

Now, on my machine, when I run python setup.py build_ext --inplace, the first step passes, and the file helloworld.c is generated, and then the gcc compilation passes as well.

现在,在我的机器上,当我运行python setup.py build_ext --inplace时,第一步通过,生成文件helloworld.c,然后gcc编译也通过。

Now if you open the file helloworld.c, you can check that your variable foo is actually declared as an int128.

现在,如果打开helloworld.c文件,可以检查变量foo是否实际声明为int128。

Be very careful with using this workaround. In particular, it can happen that cython will not require a cast in the C code if you assign an int128 to a int64 for example, because at that step of the process it actually does not distinguish between them.

使用此变通方法时要非常小心。特别是,如果你将int128分配给int64,那么cython可能不需要在C代码中使用强制转换,因为在该过程的那一步它实际上不区分它们。

#2

I'll throw my two cents in here.

我会在这里扔两分钱。

First, the solution proposed in the other answers saying to use an external typedef is not just a workaround, that is the way the Cython docs say things like this should be done. See the relevant section. Quote: "If the header file uses typedef names such as word to refer to platform-dependent flavours of numeric types, you will need a corresponding ctypedef statement, but you don’t need to match the type exactly, just use something of the right general kind (int, float, etc). For example ctypedef int word will work okay whatever the actual size of a word is (provided the header file defines it correctly). Conversion to and from Python types, if any, will also be used for this new type."

首先,在其他答案中提出的使用外部typedef的解决方案不仅仅是一种解决方法,这就是Cython文档说这样的事情应该完成的方式。请参阅相关部分。引用:“如果头文件使用typedef名称(例如word)来引用依赖于平台的数字类型,则需要相应的ctypedef语句,但是您不需要完全匹配类型,只需使用右侧的内容一般类型(int,float等)。例如ctypedef int word可以正常工作,无论单词的实际大小是多少(前提是头文件正确定义)。如果有的话,也将使用Python类型的转换(如果有的话)对于这种新型。“

Also, it isn't necessary to actually create a header file with a typedef for a type you've already included somewhere else along the way. Just do this

此外,没有必要为您已经包含在其他地方的类型创建一个带有typedef的头文件。就这样做吧

cdef extern from *:
    ctypedef int int128 "__int128_t"

Or, if you feel like keeping the name the same in Cython as it is in C,

或者,如果您想在Cython中保持名称与在C中保持相同,

cdef extern from *:
    ctypedef int __int128_t

Here's a test to demonstrate that this is working. If the 128 bit arithmetic is working, a > 1, and a is representable as a 64 bit integer, the first function will print the same number back again. If it is not, integer overflow should cause it to print 0. The second function shows what happens if 64 bit arithmetic is used.

这是一个测试,以证明这是有效的。如果128位算术正在工作,a> 1,并且a可表示为64位整数,则第一个函数将再次打印相同的数字。如果不是,整数溢出应该使它打印0.第二个函数显示如果使用64位算术会发生什么。

Cython file

# cython: cdivision = True

cdef extern from *:
    ctypedef int int128 "__int128_t"

def myfunc(long long a):
    cdef int128 i = a
    # set c to be the largest positive integer possible for a signed 64 bit integer
    cdef long long c = 0x7fffffffffffffff
    i *= c
    cdef long long b = i / c
    print b

def myfunc_bad(long long a):
    cdef long long i = a
    # set c to be the largest positive integer possible for a signed 64 bit integer
    cdef long long c = 0x7fffffffffffffff
    i *= c
    cdef long long b = i / c
    print b

In Python, after both functions have been imported, myfunc(12321) prints the correct value while myfunc_bad(12321) prints 0.

在Python中,在导入了两个函数后,myfunc(12321)打印正确的值,而myfunc_bad(12321)打印0。

#3

Here is an example for using the hack proposed by @Giulio Ghirardo.

以下是使用@Giulio Ghirardo提出的黑客的示例。

The file cbitset.px contains:

文件cbitset.px包含:

typedef unsigned __int128 bitset;

The file bitset.pyx contains:

文件bitset.pyx包含:

from libc.stdlib cimport malloc
from libc.stdio cimport printf

cdef extern from "cbitset.h":
    ctypedef unsigned long long bitset

cdef char* bitset_tostring(bitset n):
    cdef char* bitstring = <char*>malloc(8 * sizeof(bitset) * sizeof(char) + 1)
    cdef int i = 0
    while n:
        if (n & <bitset>1):
            bitstring[i] = '1'
        else:
            bitstring[i] = '0'

        n >>= <bitset>1
        i += 1
    bitstring[i] = '\0'
    return bitstring

cdef void print_bitset(bitset n):
    printf("%s\n", bitset_tostring(n))

The file main.pyx contains:

main.pyx文件包含:

from bitset cimport print_bitset

cdef extern from "cbitset.h":
    ctypedef unsigned long long bitset

# x contains a number consisting of more than 64 1's
cdef bitset x = (<bitset>1 << 70) - 1

print_bitset(x)
# 1111111111111111111111111111111111111111111111111111111111111111111111

The file setup.py contains:

setup.py文件包含:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    name="My app that used 128 bit ints",
    ext_modules=cythonize('main.pyx')
)

Compile this using the command

使用命令编译它

python3 setup.py build_ext --inplace

and run using the command

并使用该命令运行

python3 -c 'import main'