In my line of work it's hard to go five minutes without someone extolling the virtues of MQ Series or MSMQ or the like, and I always wonder, after the sparkle of buzzwords has passed, what are some actual examples of these wonderful devices out in the real world.
在我的工作中,如果没有人颂扬MQ系列或MSMQ之类的优点,很难走五分钟,而且我总是想知道,在流行语的闪光过去之后,这些精彩设备的实际例子是什么?真实世界。
What I'm looking for is something that might inspire me to find a use for one of these or give me some kind of metric I can use to evaluate a message bus/message broker/message queue -- hell, even something that will explain what the differences are between the aforementioned message* things.
我正在寻找的东西可能会激发我找到其中一个的用途或给我一些我可以用来评估消息总线/消息代理/消息队列的度量 - 地狱,甚至可以解释的东西上述消息*事物之间的区别是什么。
6 个解决方案
#1
Message queuing solutions like MQ Series or MSMQ are used extensively for integrating distributed enterprise applications, especially running on different platforms. Done right (with persistent queues, asynchronous design rather than 'RPC over MQ' and attention to operational requirements), this gives you high availability compared to synchronous request/reply integrations such as RPC or boilerplate web services (whose availability is the product of the respective availabilities: integrating ten systems with 99 % availability synchronously gives you a combined availability of no more than 90 % -- or worse if there's just one weak link). Mind you, this requires the message queues to have high availability in themselves: we use our mainframe for that purpose (take a guess at which product we're using!).
MQ系列或MSMQ等消息队列解决方案广泛用于集成分布式企业应用程序,尤其是在不同平台上运行。正确完成(使用持久性队列,异步设计而非“RPC over MQ”并注意操作要求),与RPC或样板Web服务(其可用性是其产品)的同步请求/回复集成相比,这提供了高可用性各自的可用性:同步整合十个系统和99%的可用性,使您的组合可用性不超过90% - 或者如果只有一个弱链接则更糟糕。请注意,这需要消息队列本身具有高可用性:我们使用我们的大型机来实现这一目的(猜测我们正在使用哪种产品!)。
Message (or integration) brokers and 'buses' are a more complex kettle of fish. They can add
消息(或集成)经纪人和“公共汽车”是一个更复杂的鱼。他们可以添加
- translation between different content representations (text encodings and code pages)
- supervision, detecting when a target system does not pick up queued messages and raising alarms or automatically restarting
- transformation, when systems do not 'speak the same language' and represent e.g. customer or product records differently: this can in principle help you deploy new versions at different rates
- routing (up to and including publish/subscribe) to decouple a sending system from knowledge of the details of recipients, thus reducing impact of changes in target systems
- orchestration, where you can coordinate messages between a number of systems to track a longer real-life business process (say, from customer order to delivery to invoicing).
不同内容表示之间的转换(文本编码和代码页)
监督,检测目标系统何时不接收排队的消息并发出警报或自动重启
转换,当系统不“说同一种语言”并表示例如客户或产品记录不同:原则上可以帮助您以不同的速率部署新版本
路由(直到并包括发布/订阅)以使发送系统与接收者的详细信息分离,从而减少目标系统中的更改的影响
编排,您可以在多个系统之间协调消息,以跟踪更长的实际业务流程(例如,从客户订单到交付到开票)。
I've listed these functionalities in roughly increasing order of difficulty (and potential reward). The higher you get, the more mature your organization (including the business side) needs to be in order to gain the advantages.
我已经按照大致递增的难度(和潜在的奖励)列出了这些功能。您获得的越高,您的组织(包括业务方面)需要越成熟,以获得优势。
#2
Without getting into specifics about particular products, I can give you some of the benefits to using a typical message queuing system.
在没有详细介绍特定产品的情况下,我可以为您提供使用典型消息排队系统的一些好处。
They are typically good infrastructure for simulating the publisher/subscriber pattern. Think of a big event system where you are not limited to one executable, or even one machine. You put information into these queues, such that the data can be picked up by any application who is listening for it.
它们通常是用于模拟发布者/订阅者模式的良好基础结构。想想一个大事件系统,您不仅限于一个可执行文件,甚至一台机器。您将信息放入这些队列中,以便任何正在监听它的应用程序都可以获取数据。
Most message queuing systems allow for persistent queues. Think of a typical event system. If the listener is disconnected or otherwise unresponsive at the time of the event, then the event is missed. With a persistent message queue, the message will remain in queue until the listener is reconnected. No data/events are lost in this way.
大多数消息队列系统允许持久队列。想想典型的事件系统。如果听众在事件发生时断开连接或无响应,则错过该事件。使用持久性消息队列,消息将保留在队列中,直到重新连接侦听器。以这种方式不会丢失任何数据/事件。
I don't know about the products you listed, but I know with JMS you can have fine grained control over threading as messages are popped from the queue. In theory, you could have a thread per queue, performing actions in that thread, on the messages as they are pulled off. Alternatively, you have the ability to pull messages from multiple queues and perform actions on them, all within a shared thread.
我不知道您列出的产品,但我知道使用JMS,您可以对线程进行细粒度控制,因为消息从队列中弹出。从理论上讲,您可以在每个队列中拥有一个线程,在该线程中执行操作,并在消息被删除时对其进行操作。或者,您可以从共享线程中的多个队列中提取消息并对它们执行操作。
#3
While I had a very bitter experience with MQ Series partially due to the fact that it was pushed on to us (a Microsoft shop) by the partnering company, the use of MQ Series (Or any Messaging system) was an integral part to the application.
虽然我对MQ系列有一段非常痛苦的经历,部分原因在于它由合作公司推送给我们(微软商店),但MQ系列(或任何消息系统)的使用是应用程序不可或缺的一部分。 。
Essentially we were building a process that handled supply chain fullfilment for backorder items. If our partner a distributor didn't have the items their customers wanted, they would send a message into a B2B site, that would target potential companies that could fulfil the order.
基本上我们正在构建一个处理延期交货项目供应链完整性的流程。如果我们的合作伙伴是经销商没有他们客户想要的商品,他们会向B2B网站发送消息,该网站将针对可以履行订单的潜在公司。
We had built two different flavors of integration. The first was an ftp approach where fixed width files were sent back and forth at regular intervals, and we had added all sorts of rules to help ensure we didn't miss any data.
我们构建了两种不同的集成方式。第一种是ftp方法,其中固定宽度文件定期来回发送,我们添加了各种规则以帮助确保我们不会遗漏任何数据。
The second was using MQ Series where the messages were placed into a queue using guarenteed delivery. Then we would pop the queue and process the messages. The queing system was great benefit here as it allowed us a reliable way to transmit critical messages that resulted in real money being moved around.
第二个是使用MQ Series,其中使用guarenteed传递将消息放入队列。然后我们将弹出队列并处理消息。排队系统在这里是非常有益的,因为它使我们能够以可靠的方式传输导致真钱转移的关键信息。
On the flip side with the same MQ Series we had to implement a synchronous query to get information. We wanted it to be synchronous because our users accessing this via the web would wait to get the information. Doing this over MQ Series was a very interesting and painful challenge. The only reason MQ was used here was because it was an existing line of communication and the query functionality already existed.
在具有相同MQ系列的另一方面,我们必须实现同步查询以获取信息。我们希望它是同步的,因为我们的用户通过网络访问它会等待获取信息。在MQ系列上做这件事是一个非常有趣和痛苦的挑战。这里使用MQ的唯一原因是因为它是现有的通信线路,并且查询功能已经存在。
A second example and this time was using MSMQ was a site that collected information from dialhome code injected into client applications. The dialhome code would collect feature usage statistics like Microsoft's SQM program. When the messages came in to the web service we would drop them on a queue, Then we could have any number of application servers popping the messages and pushing them to the database to be rolled into the warehouse.
第二个例子,这次使用的MSMQ是一个从注入客户端应用程序的dialhome代码中收集信息的站点。 dialhome代码将收集功能使用情况统计信息,如Microsoft的SQM程序。当消息进入Web服务时,我们会将它们放在队列中,然后我们可以让任意数量的应用程序服务器弹出消息并将它们推送到数据库以进入仓库。
MSMQ here ensured we could handle bursts of messages by quickly placing them on the queue. This help the scalability and reliability of the system.
MSMQ在这里确保我们可以通过快速将它们放在队列中来处理消息突发。这有助于系统的可扩展性和可靠性。
#4
A message queue is useful to implement load balancing. For example, the server receives "job" messages (orders, status messages ...) and distributes them to all listening clients.
消息队列对于实现负载平衡很有用。例如,服务器接收“作业”消息(订单,状态消息......)并将它们分发给所有侦听客户端。
The message queue guarantees that a message will be delivered to exactly one client.
消息队列保证将消息传递给一个客户端。
If the clients run on different computers, the total load will be distributed and it will be easy to throw another client to the message load when neccessary, the client just has to connect to the queue and will receive (some of) the messages.
如果客户端在不同的计算机上运行,则总负载将被分配,并且在必要时很容易将另一个客户端抛向消息加载,客户端只需连接到队列并将接收(某些)消息。
#5
A good queuing system makes it easier to do distributed computing over multiple threads, processors, machines, (& even organizations).
良好的排队系统使得在多个线程,处理器,机器(甚至组织)上进行分布式计算变得更加容易。
A while back (10 years) I used a message sending metaphor to implement a front-office options pricing system for an interdealer broker. We had services implemented in C++, VB6 and Excel/VBA (even using the Excel solver!!), data storage as flat files and sql, end-user applications writen in Excel and VB6, with a complex data model (market data, yield curves and vol surfaces). Asynchronous messaging (with persistent / reliable messages and pub / sub) made the whole thing work very effectively and scalably - we were able to add Tokyo and NY offices to the London infrastructure without even visiting the remote site - just a bog standard install.
前段时间(10年)我使用了一条消息发送隐喻来为交易商经纪人实施一个前台期权定价系统。我们使用C ++,VB6和Excel / VBA(甚至使用Excel解算器!!),数据存储作为平面文件和sql,最终用户应用程序在Excel和VB6中编写,具有复杂的数据模型(市场数据,产量)曲线和体积曲面)。异步消息传递(使用持久/可靠的消息和pub / sub)使整个过程非常有效和可扩展 - 我们能够将东京和纽约办事处添加到伦敦基础设施,甚至无需访问远程站点 - 只需进行标准安装。
I'm a big fan though I am surprised at how far they haven't come in the last 10 years or so.
虽然我很惊讶他们在过去10年左右没有来到这里,我感到很惊讶。
#6
One very "close to to requirement" example from real project. Who run from several years. On ActiveMQ
真实项目中一个非常“接近要求”的例子。谁跑了几年。在ActiveMQ上
1) Trade center for shipping market.
1)航运市场贸易中心。
-
Shipping company have system who knows how many packets they can ship in real time.
运输公司有系统知道他们可以实时运送多少包。
-
Every system is different (eg: language, design, etc)
每个系统都不同(例如:语言,设计等)
-
We wrote an adapter for each company "very special IT system to ActiveMQ"
我们为每家公司编写了一个适配器“非常特殊的IT系统到ActiveMQ”
-
Each adapter has a simple job to do: post when the company has free space, and at what price. (a "transport proposition")
每个适配器都有一个简单的工作要做:当公司有可用空间时以及以什么价格发布。 (“运输主张”)
-
We wrote a client who gather all "transport proposition", and display them nicely.
我们写了一个收集所有“运输命题”的客户,并很好地展示它们。
=> Ta-da. you have a cross platform/language/process system, for company who don't want to talk to each other
=> Ta-da。对于不想互相交谈的公司,您有一个跨平台/语言/流程系统
=> Ta-da 2 : If a new company wants to come in your trade system, they only have to write the adapter.
=> Ta-da 2:如果一家新公司想进入您的交易系统,他们只需要编写适配器。
#1
Message queuing solutions like MQ Series or MSMQ are used extensively for integrating distributed enterprise applications, especially running on different platforms. Done right (with persistent queues, asynchronous design rather than 'RPC over MQ' and attention to operational requirements), this gives you high availability compared to synchronous request/reply integrations such as RPC or boilerplate web services (whose availability is the product of the respective availabilities: integrating ten systems with 99 % availability synchronously gives you a combined availability of no more than 90 % -- or worse if there's just one weak link). Mind you, this requires the message queues to have high availability in themselves: we use our mainframe for that purpose (take a guess at which product we're using!).
MQ系列或MSMQ等消息队列解决方案广泛用于集成分布式企业应用程序,尤其是在不同平台上运行。正确完成(使用持久性队列,异步设计而非“RPC over MQ”并注意操作要求),与RPC或样板Web服务(其可用性是其产品)的同步请求/回复集成相比,这提供了高可用性各自的可用性:同步整合十个系统和99%的可用性,使您的组合可用性不超过90% - 或者如果只有一个弱链接则更糟糕。请注意,这需要消息队列本身具有高可用性:我们使用我们的大型机来实现这一目的(猜测我们正在使用哪种产品!)。
Message (or integration) brokers and 'buses' are a more complex kettle of fish. They can add
消息(或集成)经纪人和“公共汽车”是一个更复杂的鱼。他们可以添加
- translation between different content representations (text encodings and code pages)
- supervision, detecting when a target system does not pick up queued messages and raising alarms or automatically restarting
- transformation, when systems do not 'speak the same language' and represent e.g. customer or product records differently: this can in principle help you deploy new versions at different rates
- routing (up to and including publish/subscribe) to decouple a sending system from knowledge of the details of recipients, thus reducing impact of changes in target systems
- orchestration, where you can coordinate messages between a number of systems to track a longer real-life business process (say, from customer order to delivery to invoicing).
不同内容表示之间的转换(文本编码和代码页)
监督,检测目标系统何时不接收排队的消息并发出警报或自动重启
转换,当系统不“说同一种语言”并表示例如客户或产品记录不同:原则上可以帮助您以不同的速率部署新版本
路由(直到并包括发布/订阅)以使发送系统与接收者的详细信息分离,从而减少目标系统中的更改的影响
编排,您可以在多个系统之间协调消息,以跟踪更长的实际业务流程(例如,从客户订单到交付到开票)。
I've listed these functionalities in roughly increasing order of difficulty (and potential reward). The higher you get, the more mature your organization (including the business side) needs to be in order to gain the advantages.
我已经按照大致递增的难度(和潜在的奖励)列出了这些功能。您获得的越高,您的组织(包括业务方面)需要越成熟,以获得优势。
#2
Without getting into specifics about particular products, I can give you some of the benefits to using a typical message queuing system.
在没有详细介绍特定产品的情况下,我可以为您提供使用典型消息排队系统的一些好处。
They are typically good infrastructure for simulating the publisher/subscriber pattern. Think of a big event system where you are not limited to one executable, or even one machine. You put information into these queues, such that the data can be picked up by any application who is listening for it.
它们通常是用于模拟发布者/订阅者模式的良好基础结构。想想一个大事件系统,您不仅限于一个可执行文件,甚至一台机器。您将信息放入这些队列中,以便任何正在监听它的应用程序都可以获取数据。
Most message queuing systems allow for persistent queues. Think of a typical event system. If the listener is disconnected or otherwise unresponsive at the time of the event, then the event is missed. With a persistent message queue, the message will remain in queue until the listener is reconnected. No data/events are lost in this way.
大多数消息队列系统允许持久队列。想想典型的事件系统。如果听众在事件发生时断开连接或无响应,则错过该事件。使用持久性消息队列,消息将保留在队列中,直到重新连接侦听器。以这种方式不会丢失任何数据/事件。
I don't know about the products you listed, but I know with JMS you can have fine grained control over threading as messages are popped from the queue. In theory, you could have a thread per queue, performing actions in that thread, on the messages as they are pulled off. Alternatively, you have the ability to pull messages from multiple queues and perform actions on them, all within a shared thread.
我不知道您列出的产品,但我知道使用JMS,您可以对线程进行细粒度控制,因为消息从队列中弹出。从理论上讲,您可以在每个队列中拥有一个线程,在该线程中执行操作,并在消息被删除时对其进行操作。或者,您可以从共享线程中的多个队列中提取消息并对它们执行操作。
#3
While I had a very bitter experience with MQ Series partially due to the fact that it was pushed on to us (a Microsoft shop) by the partnering company, the use of MQ Series (Or any Messaging system) was an integral part to the application.
虽然我对MQ系列有一段非常痛苦的经历,部分原因在于它由合作公司推送给我们(微软商店),但MQ系列(或任何消息系统)的使用是应用程序不可或缺的一部分。 。
Essentially we were building a process that handled supply chain fullfilment for backorder items. If our partner a distributor didn't have the items their customers wanted, they would send a message into a B2B site, that would target potential companies that could fulfil the order.
基本上我们正在构建一个处理延期交货项目供应链完整性的流程。如果我们的合作伙伴是经销商没有他们客户想要的商品,他们会向B2B网站发送消息,该网站将针对可以履行订单的潜在公司。
We had built two different flavors of integration. The first was an ftp approach where fixed width files were sent back and forth at regular intervals, and we had added all sorts of rules to help ensure we didn't miss any data.
我们构建了两种不同的集成方式。第一种是ftp方法,其中固定宽度文件定期来回发送,我们添加了各种规则以帮助确保我们不会遗漏任何数据。
The second was using MQ Series where the messages were placed into a queue using guarenteed delivery. Then we would pop the queue and process the messages. The queing system was great benefit here as it allowed us a reliable way to transmit critical messages that resulted in real money being moved around.
第二个是使用MQ Series,其中使用guarenteed传递将消息放入队列。然后我们将弹出队列并处理消息。排队系统在这里是非常有益的,因为它使我们能够以可靠的方式传输导致真钱转移的关键信息。
On the flip side with the same MQ Series we had to implement a synchronous query to get information. We wanted it to be synchronous because our users accessing this via the web would wait to get the information. Doing this over MQ Series was a very interesting and painful challenge. The only reason MQ was used here was because it was an existing line of communication and the query functionality already existed.
在具有相同MQ系列的另一方面,我们必须实现同步查询以获取信息。我们希望它是同步的,因为我们的用户通过网络访问它会等待获取信息。在MQ系列上做这件事是一个非常有趣和痛苦的挑战。这里使用MQ的唯一原因是因为它是现有的通信线路,并且查询功能已经存在。
A second example and this time was using MSMQ was a site that collected information from dialhome code injected into client applications. The dialhome code would collect feature usage statistics like Microsoft's SQM program. When the messages came in to the web service we would drop them on a queue, Then we could have any number of application servers popping the messages and pushing them to the database to be rolled into the warehouse.
第二个例子,这次使用的MSMQ是一个从注入客户端应用程序的dialhome代码中收集信息的站点。 dialhome代码将收集功能使用情况统计信息,如Microsoft的SQM程序。当消息进入Web服务时,我们会将它们放在队列中,然后我们可以让任意数量的应用程序服务器弹出消息并将它们推送到数据库以进入仓库。
MSMQ here ensured we could handle bursts of messages by quickly placing them on the queue. This help the scalability and reliability of the system.
MSMQ在这里确保我们可以通过快速将它们放在队列中来处理消息突发。这有助于系统的可扩展性和可靠性。
#4
A message queue is useful to implement load balancing. For example, the server receives "job" messages (orders, status messages ...) and distributes them to all listening clients.
消息队列对于实现负载平衡很有用。例如,服务器接收“作业”消息(订单,状态消息......)并将它们分发给所有侦听客户端。
The message queue guarantees that a message will be delivered to exactly one client.
消息队列保证将消息传递给一个客户端。
If the clients run on different computers, the total load will be distributed and it will be easy to throw another client to the message load when neccessary, the client just has to connect to the queue and will receive (some of) the messages.
如果客户端在不同的计算机上运行,则总负载将被分配,并且在必要时很容易将另一个客户端抛向消息加载,客户端只需连接到队列并将接收(某些)消息。
#5
A good queuing system makes it easier to do distributed computing over multiple threads, processors, machines, (& even organizations).
良好的排队系统使得在多个线程,处理器,机器(甚至组织)上进行分布式计算变得更加容易。
A while back (10 years) I used a message sending metaphor to implement a front-office options pricing system for an interdealer broker. We had services implemented in C++, VB6 and Excel/VBA (even using the Excel solver!!), data storage as flat files and sql, end-user applications writen in Excel and VB6, with a complex data model (market data, yield curves and vol surfaces). Asynchronous messaging (with persistent / reliable messages and pub / sub) made the whole thing work very effectively and scalably - we were able to add Tokyo and NY offices to the London infrastructure without even visiting the remote site - just a bog standard install.
前段时间(10年)我使用了一条消息发送隐喻来为交易商经纪人实施一个前台期权定价系统。我们使用C ++,VB6和Excel / VBA(甚至使用Excel解算器!!),数据存储作为平面文件和sql,最终用户应用程序在Excel和VB6中编写,具有复杂的数据模型(市场数据,产量)曲线和体积曲面)。异步消息传递(使用持久/可靠的消息和pub / sub)使整个过程非常有效和可扩展 - 我们能够将东京和纽约办事处添加到伦敦基础设施,甚至无需访问远程站点 - 只需进行标准安装。
I'm a big fan though I am surprised at how far they haven't come in the last 10 years or so.
虽然我很惊讶他们在过去10年左右没有来到这里,我感到很惊讶。
#6
One very "close to to requirement" example from real project. Who run from several years. On ActiveMQ
真实项目中一个非常“接近要求”的例子。谁跑了几年。在ActiveMQ上
1) Trade center for shipping market.
1)航运市场贸易中心。
-
Shipping company have system who knows how many packets they can ship in real time.
运输公司有系统知道他们可以实时运送多少包。
-
Every system is different (eg: language, design, etc)
每个系统都不同(例如:语言,设计等)
-
We wrote an adapter for each company "very special IT system to ActiveMQ"
我们为每家公司编写了一个适配器“非常特殊的IT系统到ActiveMQ”
-
Each adapter has a simple job to do: post when the company has free space, and at what price. (a "transport proposition")
每个适配器都有一个简单的工作要做:当公司有可用空间时以及以什么价格发布。 (“运输主张”)
-
We wrote a client who gather all "transport proposition", and display them nicely.
我们写了一个收集所有“运输命题”的客户,并很好地展示它们。
=> Ta-da. you have a cross platform/language/process system, for company who don't want to talk to each other
=> Ta-da。对于不想互相交谈的公司,您有一个跨平台/语言/流程系统
=> Ta-da 2 : If a new company wants to come in your trade system, they only have to write the adapter.
=> Ta-da 2:如果一家新公司想进入您的交易系统,他们只需要编写适配器。