编组是什么?当某些东西被“编组”时发生了什么?

时间:2022-12-17 14:36:09

I know this question has been asked, at least here.

我知道有人问过这个问题,至少在这里。

But there wasn't a satisfactory answer, at least not to me. There is a lot of talk about marshalling as regards interoperating with unmanaged code, but what about marshalling from one thread to another, as we have to do in .NET sometimes.

但没有令人满意的答案,至少对我来说不是。关于如何对非托管代码进行互操作有很多讨论,但是如何从一个线程编组到另一个线程,就像我们在。net中有时所做的那样。

This makes me ask, what is marshalling, really? When you give a definition of marshalling, how would you define it so that it is explaining the case of interoperability, as well as the cases where you are "marshalling" between threads?

这让我问,什么是编组,真的吗?当您给出封送的定义时,您将如何定义它,以便它解释互操作性的情况,以及在线程之间“封送”的情况?

6 个解决方案

#1


74  

Computations often need to move data from one site to another, and don't have any shared memory. So one computation sends a message containing the data to the other.

计算常常需要将数据从一个站点移动到另一个站点,并且没有任何共享内存。因此,一个计算向另一个发送包含数据的消息。

How should that data, if it is arbitrarily complicated, be sent in a message?

如果数据是任意复杂的,该如何发送消息?

Marshalling is the process of converting a data field, or an entire set of related structures, into a serialized string that can be sent in a message. To marshall a binary number, one might convert it to hexadecimal digit string, if the message format must be text. If the message will carry binary data, the binary number might be converted into 4 little-endian normalized binary bytes and sent that way. Pointers are harder; one often has to convert them into an abstract reference (e.g., a "node number") that is independent of the actual memory locations.

编组是将数据字段或一组相关结构转换成可在消息中发送的序列化字符串的过程。如果消息格式必须为文本,则可以将二进制数字转换为十六进制数字字符串。如果消息将携带二进制数据,则二进制数可能被转换为4个小字节的归一化二进制字节并以这种方式发送。指针是困难;我们通常必须将它们转换为抽象引用(例如,“节点号”),与实际的内存位置无关。

Of course, if you "marshall" data, you must eventually "unmarshall", which is the process of reading the serial stream and reconstructing the transmitted data (structure).

当然,如果您“马歇尔”数据,您最终必须“unmarshall”,即读取串行流并重构传输数据(结构)的过程。

Often there are (un)marshalling routines in a library that are used to accomplish this purpose, and sometimes there are even tools that will manufacture all the calls needed on the (un)marshalling routines to send/recieve the data.

通常在库中有(un)编组例程用于实现此目的,有时甚至有工具可以生成(un)编组例程所需的所有调用来发送/接收数据。

#2


24  

Marshalling is taking data, of some form, and translating it into a separate form. It's a very generic term, and used in many places with subtle differences in meaning.

编组是获取某种形式的数据,并将其转换为单独的形式。这是一个非常通用的术语,在很多地方都有不同的含义。

For example, in .NET, the interop layer when you're working with native types "marshals" your data from the .NET type into the appropriate form to call the native method, then "marshals" the results back.

例如,在。net中,当您使用本机类型时,interop层将. net类型中的数据“封送”到适当的表单中,以调用本机方法,然后“封送”回结果。

As for "marshalling" between threads - Often, you'll need to have code to run on a different thread than the current one. For example, if you're using Windows Forms, you can't change a UI element on a threadpool thread, so you'll need to "marshal" the call back to the UI thread. This is done by creating a delegate, and passing the delegate back to the user interface thread via Control.Invoke (which uses a rather complex system to post this back to the proper synchronization context), which in turn runs the delegate on the user interface thread for you.

至于线程之间的“封送”——通常,您需要在与当前线程不同的线程上运行代码。例如,如果您正在使用Windows窗体,您不能更改threadpool线程上的UI元素,因此需要将调用“封送”回UI线程。这是通过创建一个委托完成的,并通过控件将委托传递回用户界面线程。调用(它使用一个相当复杂的系统将其返回到适当的同步上下文中),然后在用户界面线程上运行委托。

#3


12  

Wikipedia's definition is actually pretty good.

*的定义非常好。

The overall concept of marshalling is the same as "serialization:" moving from an in-memory representation (which, in a way, is like no representation at all - when something is in memory it simply "exists") to a "hard copy" representation, whether that's XML or maybe a binary stream or something. However, depending on what you're doing, it can also imply some kind of transformation or translation to a target format.

编组的总体概念与“序列化”是相同的:“从内存中表示(在某种程度上,它就像没有表示一样——当内存中有东西时,它只是“存在”)到“硬拷贝”表示,无论是XML还是二进制流或其他什么。然而,这取决于你在做什么,它也意味着某种转换或转换成目标格式。

For process marshalling: one thread doesn't simply "call" another - data has to be packaged up and "sent" from one thread to another. Marshalling is the process of packaging that data (for example, data about the method you want to call, and its parameters).

对于进程编组:一个线程不简单地“调用”另一个线程——数据必须打包并从一个线程发送到另一个线程。封送是打包数据的过程(例如,关于要调用的方法的数据及其参数)。

If you're marshalling in terms of interop, you are packaging up a method call and its parameters into a data structure that can be sent to a process/thread running the COM component. That package needs to be in a format that the COM component can understand.

如果按照互操作进行编组,则将方法调用及其参数打包为数据结构,该数据结构可以发送给运行COM组件的进程/线程。该包需要采用COM组件可以理解的格式。

#4


4  

The way I understand marshaling is that it provides a way for you to transfer data in a consistent manner across various operating environments.

我理解封送的方式是,它为您提供了一种方法,可以在不同的操作环境中以一致的方式传输数据。

In the context of marshaling data from managed to unmanaged code, it's more or less the same.

在从托管到非托管代码的封送数据的上下文中,它或多或少是相同的。

I have some data, say an array of integers or any data type of my choosing, and I want to make it available for use within my C# code after my C++ code does some operations on it.

我有一些数据,比如一个整数数组或者我选择的任何数据类型,我想让它在我的c#代码中使用,在我的c++代码对它做一些操作之后。

I can't just say "Hey, this is where the array is, do what you want" to the C# code. An array of ints in C++ may not be stored the same way as in C#. Marshaling let's us transmit this data in an environment independent manner so that either side sees the data the same exact way.

我不能只说“嘿,这是数组的位置,做你想做的”到c#代码。c++中的ints数组可能不像c#中那样存储。编组让我们以独立于环境的方式传输数据,以便任何一方都能以相同的方式查看数据。

Another example would be in networking. You usually don't call this marshaling, but if you want to transmit it over the network, you have to typically transmit it in such a way that whoever receives it interprets the data the same way you do. Your computer could represent data in little endian order, and the other could represent it in big endian order.

另一个例子是网络。通常不调用这个封送,但是如果你想通过网络传输数据,你必须以这样一种方式进行传输,即无论谁接收到数据,都要用同样的方式来解释数据。你的计算机可以用小的endian顺序来表示数据,而另一个可以用大的endian顺序来表示。

tl;dr: Marshaling provides you a way to consistently represent data across various operating environments

dr: Marshaling为您提供了一种在不同操作环境中一致表示数据的方法。

#5


3  

From Wikipedia - Marshalling (computer science):

*-编组(计算机科学):

Marshalling (similar to serialization) is the process of transforming the memory representation of an object to a data format suitable for storage or transmission. It is typically used when data must be moved between different parts of a computer program or from one program to another.

编组(类似于序列化)是将对象的内存表示转换为适合存储或传输的数据格式的过程。当数据必须在计算机程序的不同部分之间或在一个程序之间移动时,通常使用它。

In the case of calling an unmanaged function from .NET, marshalling is used to convert .NET's data into data that the unmanaged function can consume. For instance, System.String is Unicode based, but that string might need to be converted to an ANSI string to be passed into a unmanaged C function.

在从。net调用一个非托管函数的情况下,编组被用来将. net的数据转换成非托管函数可以使用的数据。例如,系统。字符串是基于Unicode的,但是该字符串可能需要转换为ANSI字符串,以传递到非托管的C函数中。

For threading, marshalling typically refers to transfer of ownership of some data from one thread to another thread. For example, a program has two threads. The first thread reads data from the network, and the second thread computes that data. After the network thread reads some data it transfers (i.e., "marshals") the data over to computation thread to process. It might do this by writing the data to a queue shared between the two threads.

对于线程,封送通常指将某些数据的所有权从一个线程转移到另一个线程。例如,一个程序有两个线程。第一个线程从网络中读取数据,第二个线程计算数据。在网络线程读取它传输的一些数据之后(例如。,“封送”)将数据传递给要处理的计算线程。它可以通过将数据写入两个线程之间共享的队列来实现这一点。

Marshalling in threading almost always involves synchronization of the data being marshalled.

线程中的封送几乎总是涉及到被封送的数据的同步。

#6


1  

It's usually used in the context of "written in an XML format" but it could be marshalled to any format.

它通常用于“以XML格式编写”的上下文,但它可以被编组成任何格式。

2.  To arrange, place, or set in methodical order.
   (from American Heritage® Dictionary of the English Language)

So it means you're arranging the data in the methodical order/format you want. Often this is in XML format.

这意味着你要按照你想要的顺序排列数据。通常这是XML格式。

#1


74  

Computations often need to move data from one site to another, and don't have any shared memory. So one computation sends a message containing the data to the other.

计算常常需要将数据从一个站点移动到另一个站点,并且没有任何共享内存。因此,一个计算向另一个发送包含数据的消息。

How should that data, if it is arbitrarily complicated, be sent in a message?

如果数据是任意复杂的,该如何发送消息?

Marshalling is the process of converting a data field, or an entire set of related structures, into a serialized string that can be sent in a message. To marshall a binary number, one might convert it to hexadecimal digit string, if the message format must be text. If the message will carry binary data, the binary number might be converted into 4 little-endian normalized binary bytes and sent that way. Pointers are harder; one often has to convert them into an abstract reference (e.g., a "node number") that is independent of the actual memory locations.

编组是将数据字段或一组相关结构转换成可在消息中发送的序列化字符串的过程。如果消息格式必须为文本,则可以将二进制数字转换为十六进制数字字符串。如果消息将携带二进制数据,则二进制数可能被转换为4个小字节的归一化二进制字节并以这种方式发送。指针是困难;我们通常必须将它们转换为抽象引用(例如,“节点号”),与实际的内存位置无关。

Of course, if you "marshall" data, you must eventually "unmarshall", which is the process of reading the serial stream and reconstructing the transmitted data (structure).

当然,如果您“马歇尔”数据,您最终必须“unmarshall”,即读取串行流并重构传输数据(结构)的过程。

Often there are (un)marshalling routines in a library that are used to accomplish this purpose, and sometimes there are even tools that will manufacture all the calls needed on the (un)marshalling routines to send/recieve the data.

通常在库中有(un)编组例程用于实现此目的,有时甚至有工具可以生成(un)编组例程所需的所有调用来发送/接收数据。

#2


24  

Marshalling is taking data, of some form, and translating it into a separate form. It's a very generic term, and used in many places with subtle differences in meaning.

编组是获取某种形式的数据,并将其转换为单独的形式。这是一个非常通用的术语,在很多地方都有不同的含义。

For example, in .NET, the interop layer when you're working with native types "marshals" your data from the .NET type into the appropriate form to call the native method, then "marshals" the results back.

例如,在。net中,当您使用本机类型时,interop层将. net类型中的数据“封送”到适当的表单中,以调用本机方法,然后“封送”回结果。

As for "marshalling" between threads - Often, you'll need to have code to run on a different thread than the current one. For example, if you're using Windows Forms, you can't change a UI element on a threadpool thread, so you'll need to "marshal" the call back to the UI thread. This is done by creating a delegate, and passing the delegate back to the user interface thread via Control.Invoke (which uses a rather complex system to post this back to the proper synchronization context), which in turn runs the delegate on the user interface thread for you.

至于线程之间的“封送”——通常,您需要在与当前线程不同的线程上运行代码。例如,如果您正在使用Windows窗体,您不能更改threadpool线程上的UI元素,因此需要将调用“封送”回UI线程。这是通过创建一个委托完成的,并通过控件将委托传递回用户界面线程。调用(它使用一个相当复杂的系统将其返回到适当的同步上下文中),然后在用户界面线程上运行委托。

#3


12  

Wikipedia's definition is actually pretty good.

*的定义非常好。

The overall concept of marshalling is the same as "serialization:" moving from an in-memory representation (which, in a way, is like no representation at all - when something is in memory it simply "exists") to a "hard copy" representation, whether that's XML or maybe a binary stream or something. However, depending on what you're doing, it can also imply some kind of transformation or translation to a target format.

编组的总体概念与“序列化”是相同的:“从内存中表示(在某种程度上,它就像没有表示一样——当内存中有东西时,它只是“存在”)到“硬拷贝”表示,无论是XML还是二进制流或其他什么。然而,这取决于你在做什么,它也意味着某种转换或转换成目标格式。

For process marshalling: one thread doesn't simply "call" another - data has to be packaged up and "sent" from one thread to another. Marshalling is the process of packaging that data (for example, data about the method you want to call, and its parameters).

对于进程编组:一个线程不简单地“调用”另一个线程——数据必须打包并从一个线程发送到另一个线程。封送是打包数据的过程(例如,关于要调用的方法的数据及其参数)。

If you're marshalling in terms of interop, you are packaging up a method call and its parameters into a data structure that can be sent to a process/thread running the COM component. That package needs to be in a format that the COM component can understand.

如果按照互操作进行编组,则将方法调用及其参数打包为数据结构,该数据结构可以发送给运行COM组件的进程/线程。该包需要采用COM组件可以理解的格式。

#4


4  

The way I understand marshaling is that it provides a way for you to transfer data in a consistent manner across various operating environments.

我理解封送的方式是,它为您提供了一种方法,可以在不同的操作环境中以一致的方式传输数据。

In the context of marshaling data from managed to unmanaged code, it's more or less the same.

在从托管到非托管代码的封送数据的上下文中,它或多或少是相同的。

I have some data, say an array of integers or any data type of my choosing, and I want to make it available for use within my C# code after my C++ code does some operations on it.

我有一些数据,比如一个整数数组或者我选择的任何数据类型,我想让它在我的c#代码中使用,在我的c++代码对它做一些操作之后。

I can't just say "Hey, this is where the array is, do what you want" to the C# code. An array of ints in C++ may not be stored the same way as in C#. Marshaling let's us transmit this data in an environment independent manner so that either side sees the data the same exact way.

我不能只说“嘿,这是数组的位置,做你想做的”到c#代码。c++中的ints数组可能不像c#中那样存储。编组让我们以独立于环境的方式传输数据,以便任何一方都能以相同的方式查看数据。

Another example would be in networking. You usually don't call this marshaling, but if you want to transmit it over the network, you have to typically transmit it in such a way that whoever receives it interprets the data the same way you do. Your computer could represent data in little endian order, and the other could represent it in big endian order.

另一个例子是网络。通常不调用这个封送,但是如果你想通过网络传输数据,你必须以这样一种方式进行传输,即无论谁接收到数据,都要用同样的方式来解释数据。你的计算机可以用小的endian顺序来表示数据,而另一个可以用大的endian顺序来表示。

tl;dr: Marshaling provides you a way to consistently represent data across various operating environments

dr: Marshaling为您提供了一种在不同操作环境中一致表示数据的方法。

#5


3  

From Wikipedia - Marshalling (computer science):

*-编组(计算机科学):

Marshalling (similar to serialization) is the process of transforming the memory representation of an object to a data format suitable for storage or transmission. It is typically used when data must be moved between different parts of a computer program or from one program to another.

编组(类似于序列化)是将对象的内存表示转换为适合存储或传输的数据格式的过程。当数据必须在计算机程序的不同部分之间或在一个程序之间移动时,通常使用它。

In the case of calling an unmanaged function from .NET, marshalling is used to convert .NET's data into data that the unmanaged function can consume. For instance, System.String is Unicode based, but that string might need to be converted to an ANSI string to be passed into a unmanaged C function.

在从。net调用一个非托管函数的情况下,编组被用来将. net的数据转换成非托管函数可以使用的数据。例如,系统。字符串是基于Unicode的,但是该字符串可能需要转换为ANSI字符串,以传递到非托管的C函数中。

For threading, marshalling typically refers to transfer of ownership of some data from one thread to another thread. For example, a program has two threads. The first thread reads data from the network, and the second thread computes that data. After the network thread reads some data it transfers (i.e., "marshals") the data over to computation thread to process. It might do this by writing the data to a queue shared between the two threads.

对于线程,封送通常指将某些数据的所有权从一个线程转移到另一个线程。例如,一个程序有两个线程。第一个线程从网络中读取数据,第二个线程计算数据。在网络线程读取它传输的一些数据之后(例如。,“封送”)将数据传递给要处理的计算线程。它可以通过将数据写入两个线程之间共享的队列来实现这一点。

Marshalling in threading almost always involves synchronization of the data being marshalled.

线程中的封送几乎总是涉及到被封送的数据的同步。

#6


1  

It's usually used in the context of "written in an XML format" but it could be marshalled to any format.

它通常用于“以XML格式编写”的上下文,但它可以被编组成任何格式。

2.  To arrange, place, or set in methodical order.
   (from American Heritage® Dictionary of the English Language)

So it means you're arranging the data in the methodical order/format you want. Often this is in XML format.

这意味着你要按照你想要的顺序排列数据。通常这是XML格式。