C - 共享内存 - 共享结构中的动态数组

i'm trying to share a struct like this
example:

我正在尝试共享一个像这个例子的结构:

typedef struct {
    int* a;
    int b;
    int c;
} ex;

between processes, the problem is that when I initialize 'a' with a malloc, it becomes private to the heap of the process that do this(or at least i think this is what happens). Is there any way to create a shared memory (with shmget, shmat) with this struct that works?

在进程之间,问题是当我使用malloc初始化'a'时,它变为私有的进程堆执行此操作(或者至少我认为这是发生的事情)。有没有办法用这个有效的结构创建共享内存(使用shmget,shmat)?

EDIT: I'm working on Linux.
EDIT: I have a process that initialize the buffer like this:

编辑:我正在研究Linux。编辑:我有一个初始化缓冲区的进程,如下所示:

key_t key = ftok("gr", 'p');   
int mid = shmget(key, sizeof(ex), IPC_CREAT | 0666);    
ex* e = NULL;
status b_status = init(&e, 8); //init gives initial values to b c and allocate space for 'a' with a malloc
e = (ex*)shmat(mid, NULL, 0);

the other process attaches himself to the shared memory like this:

另一个进程将自己附加到共享内存,如下所示:

key_t key = ftok("gr", 'p');
int shmid = shmget(key, sizeof(ex), 0);
ex* e;
e = (ex*)shmat(shmid, NULL, 0);  

and later get an element from a, in this case that in position 1

然后从a获取一个元素,在这种情况下,在位置1

int i = get_el(e, 1);

5 个解决方案

#1

The memory you allocate to a pointer using malloc() is private to that process. So, when you try to access the pointer in another process(other than the process which malloced it) you are likely going to access an invalid memory page or a memory page mapped in another process address space. So, you are likely to get a segfault.

使用malloc()分配给指针的内存对该进程是私有的。因此,当您尝试访问另一个进程中的指针(除了对其进行malloced的进程)时,您可能会访问无效的内存页面或映射到另一个进程地址空间的内存页面。所以,你可能会遇到段错误。

If you are using the shared memory, you must make sure all the data you want to expose to other processes is "in" the shared memory segment and not private memory segments of the process.

如果使用共享内存,则必须确保要向其他进程公开的所有数据都在“共享内存”段中,而不是该进程的专用内存段。

You could try, leaving the data at a specified offset in the memory segment, which can be concretely defined at compile time or placed in a field at some known location in the shared memory segment.

您可以尝试将数据保留在内存段中的指定偏移处,这可以在编译时具体定义,也可以放置在共享内存段中某个已知位置的字段中。

Eg: If you are doing this

例如:如果你这样做

char *mem = shmat(shmid2, (void*)0, 0);

char * mem = shmat(shmid2,(void *)0,0);

// So, the mystruct type is at offset 0.
mystruct *structptr = (mystruct*)mem;

// Now we have a structptr, use an offset to get some other_type.
other_type *other = (other_type*)(mem + structptr->offset_of_other_type);

Other way would be to have a fixed size buffer to pass the information using the shared memory approach, instead of using the dynamically allocated pointer.

其他方法是使用固定大小的缓冲区来使用共享内存方法传递信息,而不是使用动态分配的指针。

Hope this helps.

希望这可以帮助。

#2

First of all, to share the content pointed by your int *a field, you will need to copy the whole memory related to it. Thus, you will need a shared memory that can hold at least size_t shm_size = sizeof(struct ex) + get_the_length_of_your_ex();.

首先,要共享int * a字段指向的内容,您需要复制与其相关的整个内存。因此,您将需要一个至少可以保存size_t shm_size = sizeof(struct ex)+ get_the_length_of_your_ex();的共享内存。

From now on, since you mentioned shmget and shmat, I will assume you run a Linux system.
The first step is the shared memory segment creation. It would be a good thing if you can determine an upper bound to the size of the int *a content. This way you would not have to create/delete the shared memory segment over and over again. But if you do so, an extra overhead to state how long is the actual data will be needed. I will assume that a simple size_t will do the trick for this purpose.

从现在开始,既然你提到了shmget和shmat,我会假设你运行一个Linux系统。第一步是创建共享内存段。如果您可以确定int * a内容大小的上限,那将是一件好事。这样您就不必一遍又一遍地创建/删除共享内存段。但是如果你这样做,需要额外的开销来说明需要多长时间的实际数据。我会假设一个简单的size_t就可以达到这个目的。

Then, after you created your segment, you must set the data correctly to make it hold what you want. Notice that while the physical address of the memory segment is always the same, when calling shmat you will get virtual pointers, which are only usable in the process that called shmat. The example code below should give you some tricks to do so.

然后,在创建细分后,您必须正确设置数据以使其保持您想要的状态。请注意,虽然内存段的物理地址始终相同,但在调用shmat时,您将获得虚拟指针,这些指针仅在调用shmat的进程中可用。下面的示例代码应该为您提供一些技巧。

#include <sys/types.h>
#include <sys/ipc.h>

/* Assume a cannot point towards an area larger than 4096 bytes. */
#define A_MAX_SIZE (size_t)4096

struct ex {
    int *a;
    int b;
    int c;
}

int shm_create(void)
{
    /*
     * If you need to share other structures,
     * You'll need to pass the key_t as an argument
     */
    key_t k = ftok("/a/path/of/yours");
    int shm_id = 0;
    if (0 > (shm_id = shmget(
        k, sizeof(struct ex) + A_MAX_SIZE + sizeof(size_t), IPC_CREAT|IPC_EXCL|0666))) {
        /* An error occurred, add desired error handling. */
    }
    return shm_id;
}

/*
 * Fill the desired shared memory segment with the structure
 */
int shm_fill(int shmid, struct ex *p_ex)
{
    void *p = shmat(shmid, NULL, 0);
    void *tmp = p;
    size_t data_len = get_my_ex_struct_data_len(p_ex);
    if ((void*)(-1) == p) {
        /* Add desired error handling */
        return -1;
    }
    memcpy(tmp, p_ex, sizeof(struct ex));
    tmp += sizeof(struct ex);
    memcpy(tmp, &data_len, sizeof(size_t);
    tmp += 4;
    memcpy(tmp, p_ex->a, data_len);

    shmdt(p);
    /*
     * If you want to keep the reference so that
     * When modifying p_ex anywhere, you update the shm content at the same time :
     * - Don't call shmdt()
     * - Make p_ex->a point towards the good area :
     * p_ex->a = p + sizeof(struct ex) + sizeof(size_t);
     * Never ever modify a without detaching the shm ...
     */
    return 0;
}

/* Get the ex structure from a shm segment */
int shm_get_ex(int shmid, struct ex *p_dst)
{
    void *p = shmat(shmid, NULL, SHM_RDONLY);
    void *tmp;
    size_t data_len = 0;
    if ((void*)(-1) == p) {
        /* Error ... */
        return -1;
    }
    data_len = *(size_t*)(p + sizeof(struct ex))
    if (NULL == (tmp = malloc(data_len))) {
        /* No memory ... */
        shmdt(p);
        return -1;
    }
    memcpy(p_dst, p, sizeof(struct ex));
    memcpy(tmp, (p + sizeof(struct ex) + sizeof(size_t)), data_len);
    p_dst->a = tmp;
    /*
     * If you want to modify "globally" the structure,
     * - Change SHM_RDONLY to 0 in the shmat() call
     * - Make p_dst->a point to the good offset :
     * p_dst->a = p + sizeof(struct ex) + sizeof(size_t);
     * - Remove from the code above all the things made with tmp (malloc ...)
     */
    return 0;
}

/*
 * Detach the given p_ex structure from a shm segment.
 * This function is useful only if you use the shm segment
 * in the way I described in comment in the other functions.
 */
void shm_detach_struct(struct ex *p_ex)
{
    /*
     * Here you could : 
     * - alloc a local pointer
     * - copy the shm data into it
     * - detach the segment using the current p_ex->a pointer
     * - assign your local pointer to p_ex->a
     * This would save locally the data stored in the shm at the call
     * Or if you're lazy (like me), just detach the pointer and make p_ex->a = NULL;
     */

    shmdt(p_ex->a - sizeof(struct ex) - sizeof(size_t));
    p_ex->a = NULL;
}

Excuse my laziness, it would be space-optimized to not copy at all the value of the int *a pointer of the struct ex since it is completely unused in the shared memory, but I spared myself extra-code to handle this (and some pointer checkings like the p_ex arguments integrity).

请原谅我的懒惰,它将进行空间优化,不会复制结构ex的指针int *的所有值,因为它在共享内存中完全未使用,但我不遗余力地处理这个(以及一些指针检查,如p_ex参数完整性)。

But when you are done, you must find a way to share the shm ID between your processes. This could be done using sockets, pipes ... Or using ftok with the same input.

但是当你完成后,你必须找到一种方法来在你的进程之间共享shm ID。这可以使用套接字,管道...或使用具有相同输入的ftok来完成。

#3

Are you working in Windows or Linux? In any case what you need is a memory mapped file. Documentation with code examples here,

你在Windows或Linux工作吗?在任何情况下,您需要的是内存映射文件。这里带有代码示例的文档,

http://msdn.microsoft.com/en-us/library/aa366551%28VS.85%29.aspx http://menehune.opt.wfu.edu/Kokua/More_SGI/007-2478-008/sgi_html/ch03.html

#4

You need to use shared memory/memory mapped files/whatever your OS gives you. In general, IPC and sharing memory between processes is quite OS dependent, especially in low-level languages like C (higher-level languages usually have libraries for that - for example, even C++ has support for it using boost). If you are on Linux, I usually use shmat for small amount, and mmap (http://en.wikipedia.org/wiki/Mmap) for larger amounts. On Win32, there are many approaches; the one I prefer is usually using page-file backed memory mapped files (http://msdn.microsoft.com/en-us/library/ms810613.aspx)

您需要使用共享内存/内存映射文件/您的操作系统提供的任何内容。一般来说,IPC和进程之间共享内存非常依赖于操作系统,特别是在像C这样的低级语言中(高级语言通常有用于此的库 - 例如,即使C ++也支持使用boost)。如果您使用的是Linux,我通常会少量使用shmat,而mmap(http://en.wikipedia.org/wiki/Mmap)则需要更大的数量。在Win32上,有很多方法;我更喜欢的是通常使用页面文件支持的内存映射文件(http://msdn.microsoft.com/en-us/library/ms810613.aspx)

Also, you need to pay attention to where you are using these mechanism inside your data structures: as mentioned in the comments, without using precautions the pointer you have in your "source" process is invalid in the "target" process, and needs to be replaced/adjusted (IIRC, pointers coming from mmap are already OK(mapped); at least, under windows pointers you get out of MapViewOfFile are OK).

此外,您需要注意在数据结构中使用这些机制的位置:如注释中所述,不使用预防措施,“源”过程中的指针在“目标”过程中无效,并且需要被替换/调整(IIRC,来自mmap的指针已经可以(映射);至少,在windows指针下你可以从MapViewOfFile中获得)。

EDIT: from your edited example: What you do here:

编辑:从你编辑的例子:你在这里做什么:

e = (ex*)shmat(mid, NULL, 0);

(other process)

int shmid = shmget(key, sizeof(ex), 0);
ex* e = (ex*)shmat(shmid, NULL, 0);

is correcty, but you need to do it for each pointer you have, not only for the "main" pointer to the struct. E.g. you need to do:

是正确的,但是您需要为每个指针执行此操作,而不仅仅是指向结构的“主”指针。例如。你需要做的:

e->a = (int*)shmat(shmget(another_key, dim_of_a, IPC_CREAT | 0666), NULL, 0);

instead of creating the array with malloc. Then, on the other process, you also need to do shmget/shmat for the pointer. This is why, in the comments, I said that I usually prefer to pack the structs: so I do not need to go through the hassle to to these operations for every pointer.

而不是使用malloc创建数组。然后,在另一个进程中,您还需要为指针执行shmget / shmat。这就是为什么在评论中我说我通常更喜欢打包结构:所以我不需要为每个指针经历麻烦来进行这些操作。

#5

Convert the struct:

转换结构:

typedef struct {
     int b;
     int c;
     int a[];
} ex;

and then on parent process:

然后在父进程上:

int mid = shmget(key, sizeof(ex) + arraysize*sizeof(int), 0666);

it should work.

它应该工作。

In general, it is difficult to work with dynamic arrays inside structs in c, but in this way you are able to allocate the proper memory (this will also work in malloc: How to include a dynamic array INSIDE a struct in C?)

一般来说,很难在c中的结构中使用动态数组,但是这样你就可以分配适当的内存(这也适用于malloc:如何在C中包含动态数组INSIDE结构?)