I have a 1-D numpy array in memory
我在内存中有一个n-n numpy数组
>>> x = np.arange(5)
I want to share this data with a separate and independent (not forked) C process on the same computer using shared memory.
我想在使用共享内存的同一台计算机上使用单独且独立(非分叉)的C进程共享此数据。
I expect to do something like the following:
我期望做类似以下的事情:
- Allocate a new block of shared memory from within Python
- Copy current data into that block
- Get dtype, length, and global address of the array and pass these to a C process (for now, we'll pass to the C process over a command line interface)
- Create an appropriately typed pointer within C to the memory address
- Do some trivial computation in C
从Python中分配新的共享内存块
将当前数据复制到该块中
获取数组的dtype,length和全局地址,并将它们传递给C进程(现在,我们将通过命令行界面传递给C进程)
在C内创建一个适当的类型指针到内存地址
在C中做一些微不足道的计算
What is the best way to acheive these steps? There appear to be several Python solutions to allocate array data in shared memory. All examples I can find involve sharing between two Python processes rather than between Python and another language.
实现这些步骤的最佳方法是什么?似乎有几种Python解决方案可以在共享内存中分配数组数据。我能找到的所有例子都涉及两个Python进程之间的共享,而不是Python和另一种语言之间的共享
Minimal examples are greatly welcomed.
最受欢迎的例子非常受欢迎。
1 个解决方案
#1
2
Here is a minimal example:
这是一个最小的例子:
Python
import os
import posix_ipc
import numpy as np
x = np.arange(1000, dtype='i4')
f = posix_ipc.SharedMemory('test', flags=posix_ipc.O_CREAT, size=x.nbytes, read_only=False)
ff = os.fdopen(f.fd, mode='wb')
ff.write(x.data)
ff.close() # flush doesn't work, but this does.
C
// shm.c
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
int main(int argc, char *argv[])
{
int i, fd;
int *data;
fd = shm_open("test", O_RDONLY, 0);
if (fd == -1)
printf("Error!, bad file desciptor");
data = mmap(NULL, sizeof(int), PROT_READ, MAP_PRIVATE, fd, 0);
if (data == MAP_FAILED)
printf("Error!, map failed!");
for(i = 0; i < 1000; i++){
printf("%d, ", (int)data[i]);
}
return 0;
}
$ gcc shm.c -lrt -o shm
$ ./shm
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...
#1
2
Here is a minimal example:
这是一个最小的例子:
Python
import os
import posix_ipc
import numpy as np
x = np.arange(1000, dtype='i4')
f = posix_ipc.SharedMemory('test', flags=posix_ipc.O_CREAT, size=x.nbytes, read_only=False)
ff = os.fdopen(f.fd, mode='wb')
ff.write(x.data)
ff.close() # flush doesn't work, but this does.
C
// shm.c
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
int main(int argc, char *argv[])
{
int i, fd;
int *data;
fd = shm_open("test", O_RDONLY, 0);
if (fd == -1)
printf("Error!, bad file desciptor");
data = mmap(NULL, sizeof(int), PROT_READ, MAP_PRIVATE, fd, 0);
if (data == MAP_FAILED)
printf("Error!, map failed!");
for(i = 0; i < 1000; i++){
printf("%d, ", (int)data[i]);
}
return 0;
}
$ gcc shm.c -lrt -o shm
$ ./shm
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...