I'm building a client-server (c#) application that uses a web services to synchronize the data.
我正在构建一个客户端 - 服务器(c#)应用程序,它使用Web服务来同步数据。
Basically I am passing the XML of a DataSet back and forth. But depending on various parameters the dataset can actually be quite large. I'm compressing the XML using Gzip and want to pass that to the web server and the get the resulting compressed XML back.
基本上我来回传递DataSet的XML。但是根据各种参数,数据集实际上可能非常大。我正在使用Gzip压缩XML并希望将其传递给Web服务器并获取生成的压缩XML。
What is the best way to pass potentially large chunks of data back and forth?
来回传递潜在大块数据的最佳方法是什么?
Clarification: I guess I'm asking what format is the best to pass the data. JSON, SOAP, ordinary POST (I'm not very familiar with Web Services so I'm sure there's more that I'm not thinking of).
澄清:我想我问的是哪种格式最适合传递数据。 JSON,SOAP,普通POST(我对Web服务不太熟悉,所以我确信还有更多我不想的)。
6 个解决方案
#1
Best probably depends on a lot of factors. If by best you mean most performant, here are some points to consider:
最好的可能取决于很多因素。如果最好的话,你的意思是最高效的,这里有一些要考虑的要点:
- XML is not the best way at all
- Binary serialization is far more efficient.
- You may not have this option, however, if you need to concern yourself with interoperability. In that case, you may want to consider using a flat file or delimited format.
- If neither of those are doable, then you might consider just sending what has changed instead of all of the data.
- If that isn't an option, then compressing the XML is an... ok solution.
XML根本不是最好的方式
二进制序列化效率更高。
但是,如果您需要关注互操作性,则可能没有此选项。在这种情况下,您可能需要考虑使用平面文件或分隔格式。
如果这些都不可行,那么您可以考虑发送已更改的内容而不是所有数据。
如果这不是一个选项,那么压缩XML是一个......好的解决方案。
I'd use XML as a last resort, since there's a lot of unnecessary metadata that is included in XML that will easily increase the size of your payload by more than 5X.
我使用XML作为最后的手段,因为XML中包含了许多不必要的元数据,这些元数据很容易将有效负载的大小增加5倍以上。
#2
What you probably want to investigate is using MTOM. This allows you to send binary SOAP in a chunked manner. This is cool because chunking the data, coupled with an AJAX UI, will allow you to present a progress bar to the user that is uploading it.
您可能想要调查的是使用MTOM。这允许您以分块方式发送二进制SOAP。这很酷,因为通过分块数据和AJAX UI,您可以向正在上传它的用户显示进度条。
Here is an example of using MTOM to send/receive data in a chunked manner. I hope this helps.
以下是使用MTOM以分块方式发送/接收数据的示例。我希望这有帮助。
#3
If you use WCF for your web services you can use MTOM to send data over as a binary attachment. See:
如果您使用WCF作为Web服务,则可以使用MTOM作为二进制附件发送数据。看到:
What is the best way to send large data from WCF service to Client?
将大数据从WCF服务发送到客户端的最佳方法是什么?
#4
You can try compressing the file and sending it as binary via webservice. also, don't forget to modify your upload limits.
您可以尝试压缩文件并通过webservice将其作为二进制文件发送。另外,不要忘记修改上传限制。
how to send binary data via webservice: http://support.microsoft.com/kb/318425
如何通过webservice发送二进制数据:http://support.microsoft.com/kb/318425
#5
If all you want is to apply gzip compression to your xml data you might consider enabling transparent gzip compression at the http transport level (Accept-encoding: gzip, Content-Encoding: gzip) and let the iis handle it.
see http://weblogs.asp.net/owscott/archive/2004/01/12/IIS-Compression-in-IIS6.0.aspx#58745
如果您只想将gzip压缩应用于xml数据,您可以考虑在http传输级别启用透明gzip压缩(Accept-encoding:gzip,Content-Encoding:gzip)并让iis处理它。见http://weblogs.asp.net/owscott/archive/2004/01/12/IIS-Compression-in-IIS6.0.aspx#58745
#6
For web services, you have multiple concepts and possibilities that you can consider.
对于Web服务,您可以考虑多种概念和可能性。
Do you need multiple clients managing the same data set? Concurrently?
您是否需要多个客户端管理相同的数据集?同时?
Do you have a disconnected set where it is managed on both ends or just one manages the other views/stores/reports?
您是否有一个断开连接的设置,它在两端管理或只管理其他视图/存储/报告?
Are you simply using the client to use as an interface to the data set, and if so, would it make sense to use a different UI (web) to manage different parts of the dataset independently as a web UI based application?
您是仅仅使用客户端作为数据集的接口,如果是这样,使用不同的UI(Web)作为基于Web UI的应用程序独立管理数据集的不同部分是否有意义?
Are you making incremental changes in a transactional type interaction? (One end sends the big dataset to client and transactional changes are returned based on that?)
您是否在事务类型交互中进行增量更改? (一端将大数据集发送到客户端,并根据该返回事务更改?)
Difficult to recommend specifically without some thoughts about if/how you might dissect the dataset for more optimal management or knowing more of the specifics here.
如果没有一些关于是否/如何剖析数据集以进行更优化管理或了解更多具体细节的想法,很难具体推荐。
If you HAVE to send the large data entity, then it just becomes a matter of which method you wish to use for your pipeline, its simply and upload/download and you manage the endpoints appropriately.
如果您必须发送大型数据实体,那么您只需要将哪种方法用于管道,简单地上传/下载并适当地管理端点。
For some random thoughts:
对于一些随意的想法:
If you want to manipulate the data in the DOM, consider JSON, which is quite compatible with JavaScript.
如果您想操纵DOM中的数据,请考虑JSON,它与JavaScript完全兼容。
If you desire the XML dataset manipulation, you could place zip or some other compression if you need to pass the entire dataset, just wire it up on each end.
如果您需要XML数据集操作,则可以在需要传递整个数据集时放置zip或其他压缩,只需将其连接到每一端即可。
#1
Best probably depends on a lot of factors. If by best you mean most performant, here are some points to consider:
最好的可能取决于很多因素。如果最好的话,你的意思是最高效的,这里有一些要考虑的要点:
- XML is not the best way at all
- Binary serialization is far more efficient.
- You may not have this option, however, if you need to concern yourself with interoperability. In that case, you may want to consider using a flat file or delimited format.
- If neither of those are doable, then you might consider just sending what has changed instead of all of the data.
- If that isn't an option, then compressing the XML is an... ok solution.
XML根本不是最好的方式
二进制序列化效率更高。
但是,如果您需要关注互操作性,则可能没有此选项。在这种情况下,您可能需要考虑使用平面文件或分隔格式。
如果这些都不可行,那么您可以考虑发送已更改的内容而不是所有数据。
如果这不是一个选项,那么压缩XML是一个......好的解决方案。
I'd use XML as a last resort, since there's a lot of unnecessary metadata that is included in XML that will easily increase the size of your payload by more than 5X.
我使用XML作为最后的手段,因为XML中包含了许多不必要的元数据,这些元数据很容易将有效负载的大小增加5倍以上。
#2
What you probably want to investigate is using MTOM. This allows you to send binary SOAP in a chunked manner. This is cool because chunking the data, coupled with an AJAX UI, will allow you to present a progress bar to the user that is uploading it.
您可能想要调查的是使用MTOM。这允许您以分块方式发送二进制SOAP。这很酷,因为通过分块数据和AJAX UI,您可以向正在上传它的用户显示进度条。
Here is an example of using MTOM to send/receive data in a chunked manner. I hope this helps.
以下是使用MTOM以分块方式发送/接收数据的示例。我希望这有帮助。
#3
If you use WCF for your web services you can use MTOM to send data over as a binary attachment. See:
如果您使用WCF作为Web服务,则可以使用MTOM作为二进制附件发送数据。看到:
What is the best way to send large data from WCF service to Client?
将大数据从WCF服务发送到客户端的最佳方法是什么?
#4
You can try compressing the file and sending it as binary via webservice. also, don't forget to modify your upload limits.
您可以尝试压缩文件并通过webservice将其作为二进制文件发送。另外,不要忘记修改上传限制。
how to send binary data via webservice: http://support.microsoft.com/kb/318425
如何通过webservice发送二进制数据:http://support.microsoft.com/kb/318425
#5
If all you want is to apply gzip compression to your xml data you might consider enabling transparent gzip compression at the http transport level (Accept-encoding: gzip, Content-Encoding: gzip) and let the iis handle it.
see http://weblogs.asp.net/owscott/archive/2004/01/12/IIS-Compression-in-IIS6.0.aspx#58745
如果您只想将gzip压缩应用于xml数据,您可以考虑在http传输级别启用透明gzip压缩(Accept-encoding:gzip,Content-Encoding:gzip)并让iis处理它。见http://weblogs.asp.net/owscott/archive/2004/01/12/IIS-Compression-in-IIS6.0.aspx#58745
#6
For web services, you have multiple concepts and possibilities that you can consider.
对于Web服务,您可以考虑多种概念和可能性。
Do you need multiple clients managing the same data set? Concurrently?
您是否需要多个客户端管理相同的数据集?同时?
Do you have a disconnected set where it is managed on both ends or just one manages the other views/stores/reports?
您是否有一个断开连接的设置,它在两端管理或只管理其他视图/存储/报告?
Are you simply using the client to use as an interface to the data set, and if so, would it make sense to use a different UI (web) to manage different parts of the dataset independently as a web UI based application?
您是仅仅使用客户端作为数据集的接口,如果是这样,使用不同的UI(Web)作为基于Web UI的应用程序独立管理数据集的不同部分是否有意义?
Are you making incremental changes in a transactional type interaction? (One end sends the big dataset to client and transactional changes are returned based on that?)
您是否在事务类型交互中进行增量更改? (一端将大数据集发送到客户端,并根据该返回事务更改?)
Difficult to recommend specifically without some thoughts about if/how you might dissect the dataset for more optimal management or knowing more of the specifics here.
如果没有一些关于是否/如何剖析数据集以进行更优化管理或了解更多具体细节的想法,很难具体推荐。
If you HAVE to send the large data entity, then it just becomes a matter of which method you wish to use for your pipeline, its simply and upload/download and you manage the endpoints appropriately.
如果您必须发送大型数据实体,那么您只需要将哪种方法用于管道,简单地上传/下载并适当地管理端点。
For some random thoughts:
对于一些随意的想法:
If you want to manipulate the data in the DOM, consider JSON, which is quite compatible with JavaScript.
如果您想操纵DOM中的数据,请考虑JSON,它与JavaScript完全兼容。
If you desire the XML dataset manipulation, you could place zip or some other compression if you need to pass the entire dataset, just wire it up on each end.
如果您需要XML数据集操作,则可以在需要传递整个数据集时放置zip或其他压缩,只需将其连接到每一端即可。