用于。net的BLOBs的分布式存储?

时间:2022-07-04 16:52:36

I am looking for a reasonably well tested library+server to store a persistent distributed hash table.

我正在寻找一个经过充分测试的库+服务器来存储一个持久的分布式散列表。

I am hesistant to use SQL-based solutions as the data is highly document oriented, consisting of millions of ~64KB blobs with only a single index (computed by hash of said BLOB) - and needs to be able to be distributed for long term scaling prospects.

我喜欢使用基于sql的解决方案,因为数据是高度面向文档的,由数百万~64KB的blobs组成,只有一个索引(由上述BLOB的散列计算),并且需要能够进行长期的扩展。

Due to expense and bandwidth considerations, external solutions such as S3 are not an option.

由于费用和带宽的考虑,像S3这样的外部解决方案不是一个选项。

Something like CouchDB or Project Voldemort would be ideal - however there is a noticable lack of .NET bindings for both (PV can be IKVMC'd from Java - however has "issues".). Both key and value are byte arrays (key is 16 byte, the value is up to 2048KB averaging 64KB)

像CouchDB或Project Voldemort这样的东西将是理想的——然而,这两者都明显缺乏。net绑定(PV可以是来自Java的IKVMC),但也存在“问题”。键和值都是字节数组(键为16字节,值为2048KB,平均64KB)

I have searched so far for some kind of .NET port of Dynamo, Chord and similar - however the majority of results appear to be purely in-memory caches and lack any form of persistence or replication.

到目前为止,我已经搜索了一些关于Dynamo、Chord等类似的。net端口。然而,大多数结果似乎是纯粹的内存缓存,缺乏任何形式的持久性或复制。

Anyone got any ideas or suggestions?

有人有什么想法或建议吗?

5 个解决方案

#1


8  

Take a look at Ayende's Rhino DHT. Might be more inline with what you are looking for. The source can be acquired here.

看看艾恩德的犀牛DHT。可能更符合您正在寻找的内容。可以在这里获取源代码。

#2


2  

DryadLINQ or Hadoop.Net may help.

DryadLINQ或Hadoop。网络可以帮助。

Hadoop.Net is dotnet version of Hadoop. More about Hadoop can be found here

Hadoop。Net是Hadoop的dotnet版本。更多关于Hadoop的信息可以在这里找到

#3


2  

I actually think you should consider SQL Server 2008. Store the data in a table with a varbinary(max) column, along with a column that contains the hash of that column. Index the hash, as you suggested.

我认为您应该考虑SQL Server 2008。将数据存储在包含varbinary(max)列的表中,以及包含该列哈希的列。索引散列,如您所建议的。

You'll then be able to use the various distribution features of the product.

然后您将能够使用产品的各种分布特性。

#4


1  

Consider MS Velocity.

考虑女士的速度。

Summary: “Velocity” is a distributed in-memory application cache platform for developing scalable, available, and high-performance applications. “Velocity” fuses memory across multiple computers to give a single unified cache view to applications. Applications can store any serializable CLR object without worrying about where the object gets stored. Scalability can be achieved by simply adding more computers on demand. “Velocity” also allows for copies of data to be stored across the cluster, thus protecting data against failures. “Velocity” can be configured to run as a service accessed over the network or can be run embedded with the distributed application.

概述:“Velocity”是用于开发可伸缩、可用和高性能应用程序的分布式内存应用程序缓存平台。“Velocity”融合多个计算机的内存,为应用程序提供统一的缓存视图。应用程序可以存储任何可序列化的CLR对象,而不必担心对象的存储位置。只要按需添加更多的计算机,就可以实现可扩展性。“Velocity”还允许跨集群存储数据副本,从而保护数据不受失败的影响。“Velocity”可以配置为通过网络访问的服务,或者可以在分布式应用程序中运行。

#5


1  

You can try StorageEdge it has NCache technology at it's back-end so by using it you'll have the support of distributed cache which will boost SharePoint performance, reliability, scalability and optimize its storage at the same time.

您可以尝试StorageEdge它的后端有NCache技术,通过使用它,您将获得分布式缓存的支持,这将提高SharePoint的性能、可靠性、可扩展性,并同时优化其存储。

Here is link to StorageEdge's homepage http://www.alachisoft.com/storageedge/ I hope it help :)

下面是StorageEdge的主页http://www.alachisoft.com/storageedge/我希望它能有所帮助:)

Rest for .NET you can always try NCache a big name in Distributed Caching you can find its details on http://www.alachisoft.com/ncache/

关于。net的Rest,您可以在分布式缓存中尝试NCache,您可以在http://www.alachisoft.com/ncache/上找到它的详细信息

#1


8  

Take a look at Ayende's Rhino DHT. Might be more inline with what you are looking for. The source can be acquired here.

看看艾恩德的犀牛DHT。可能更符合您正在寻找的内容。可以在这里获取源代码。

#2


2  

DryadLINQ or Hadoop.Net may help.

DryadLINQ或Hadoop。网络可以帮助。

Hadoop.Net is dotnet version of Hadoop. More about Hadoop can be found here

Hadoop。Net是Hadoop的dotnet版本。更多关于Hadoop的信息可以在这里找到

#3


2  

I actually think you should consider SQL Server 2008. Store the data in a table with a varbinary(max) column, along with a column that contains the hash of that column. Index the hash, as you suggested.

我认为您应该考虑SQL Server 2008。将数据存储在包含varbinary(max)列的表中,以及包含该列哈希的列。索引散列,如您所建议的。

You'll then be able to use the various distribution features of the product.

然后您将能够使用产品的各种分布特性。

#4


1  

Consider MS Velocity.

考虑女士的速度。

Summary: “Velocity” is a distributed in-memory application cache platform for developing scalable, available, and high-performance applications. “Velocity” fuses memory across multiple computers to give a single unified cache view to applications. Applications can store any serializable CLR object without worrying about where the object gets stored. Scalability can be achieved by simply adding more computers on demand. “Velocity” also allows for copies of data to be stored across the cluster, thus protecting data against failures. “Velocity” can be configured to run as a service accessed over the network or can be run embedded with the distributed application.

概述:“Velocity”是用于开发可伸缩、可用和高性能应用程序的分布式内存应用程序缓存平台。“Velocity”融合多个计算机的内存,为应用程序提供统一的缓存视图。应用程序可以存储任何可序列化的CLR对象,而不必担心对象的存储位置。只要按需添加更多的计算机,就可以实现可扩展性。“Velocity”还允许跨集群存储数据副本,从而保护数据不受失败的影响。“Velocity”可以配置为通过网络访问的服务,或者可以在分布式应用程序中运行。

#5


1  

You can try StorageEdge it has NCache technology at it's back-end so by using it you'll have the support of distributed cache which will boost SharePoint performance, reliability, scalability and optimize its storage at the same time.

您可以尝试StorageEdge它的后端有NCache技术,通过使用它,您将获得分布式缓存的支持,这将提高SharePoint的性能、可靠性、可扩展性,并同时优化其存储。

Here is link to StorageEdge's homepage http://www.alachisoft.com/storageedge/ I hope it help :)

下面是StorageEdge的主页http://www.alachisoft.com/storageedge/我希望它能有所帮助:)

Rest for .NET you can always try NCache a big name in Distributed Caching you can find its details on http://www.alachisoft.com/ncache/

关于。net的Rest,您可以在分布式缓存中尝试NCache,您可以在http://www.alachisoft.com/ncache/上找到它的详细信息