I have logic consisting of selecting a large number of records from one system, performing multiple transformations (based on business rules) and inserting them into another system.
我的逻辑包括从一个系统中选择大量记录,执行多个转换(基于业务规则)并将它们插入另一个系统。
It seems like a high performance (and memory) hit to instantiate each of these records as an object, perform transformations on them and then insert all of these object into the other system.
看起来像是一个高性能(和内存)命中来将这些记录中的每一个实例化为一个对象,对它们执行转换然后将所有这些对象插入到另一个系统中。
Is the best way to achieve this in DDD to skip the classes/objects and do it straight through SQL, maybe a stored procedure?
在DDD中实现此目的的最佳方法是跳过类/对象并直接通过SQL执行,也许是存储过程?
Is there a better way using DDD to achieve this goal?
有没有更好的方法使用DDD来实现这一目标?
Note: The systems use SQL databases, at the moment object stores like CouchDB are not an option.
注意:系统使用SQL数据库,目前像CouchDB这样的对象存储不是一个选项。
1 个解决方案
#1
5
A lot of distributed systems built on DDD are using an Event-Driven Architecture, where rather than waiting to perform all the transformations in one batch, as each entity undergoes the state change that would cause it to be transformed by your system, the entity raises an event that gets published to a message bus of some kind (e.g. Mule for Java, MassTransit for .NET). Your transformation system will subscribe to this events, and as each message arrives in your system, it will perform the transformation on the entity identified in the message and then publish another message to the destination system.
许多基于DDD构建的分布式系统都使用事件驱动架构,而不是等待在一个批处理中执行所有转换,因为每个实体都经历状态更改,导致系统对其进行转换,实体引发发布到某种消息总线的事件(例如Mule for Java,MassTransit for .NET)。您的转换系统将订阅此事件,并且当每条消息到达您的系统时,它将对消息中标识的实体执行转换,然后将另一条消息发布到目标系统。
This kind of "trickle processing" can run continuously, all day long without putting the kind of load on your system that would necessitate the job being run after-hours. If you're concerned about performance, this kind of architecture might result in a system that has the last record transformed 5 minutes after COB, where a batch job might not even be able to run until 3 am (after all the other batch jobs have finished).
这种“涓流处理”可以整天持续运行,而不会给您的系统带来任何负载,这将使工作在下班后运行。如果您担心性能,这种架构可能会导致系统在COB后5分钟转换最后一条记录,批处理作业甚至可能无法运行到凌晨3点(在所有其他批处理作业之后)完成)。
If you truly don't want the target system to be updated until midnight, e.g., just queue the messages up until midnight, and then publish them to the destination system's endpoint.
如果您确实不希望目标系统在午夜之前更新,例如,只需将消息排队到午夜,然后将它们发布到目标系统的端点。
Greg Young has blogged and presented extensively on this kind of architecture. Check out his work on InfoQ.
Greg Young在博客上发表了大量有关此类架构的文章。看看他在InfoQ上的工作。
#1
5
A lot of distributed systems built on DDD are using an Event-Driven Architecture, where rather than waiting to perform all the transformations in one batch, as each entity undergoes the state change that would cause it to be transformed by your system, the entity raises an event that gets published to a message bus of some kind (e.g. Mule for Java, MassTransit for .NET). Your transformation system will subscribe to this events, and as each message arrives in your system, it will perform the transformation on the entity identified in the message and then publish another message to the destination system.
许多基于DDD构建的分布式系统都使用事件驱动架构,而不是等待在一个批处理中执行所有转换,因为每个实体都经历状态更改,导致系统对其进行转换,实体引发发布到某种消息总线的事件(例如Mule for Java,MassTransit for .NET)。您的转换系统将订阅此事件,并且当每条消息到达您的系统时,它将对消息中标识的实体执行转换,然后将另一条消息发布到目标系统。
This kind of "trickle processing" can run continuously, all day long without putting the kind of load on your system that would necessitate the job being run after-hours. If you're concerned about performance, this kind of architecture might result in a system that has the last record transformed 5 minutes after COB, where a batch job might not even be able to run until 3 am (after all the other batch jobs have finished).
这种“涓流处理”可以整天持续运行,而不会给您的系统带来任何负载,这将使工作在下班后运行。如果您担心性能,这种架构可能会导致系统在COB后5分钟转换最后一条记录,批处理作业甚至可能无法运行到凌晨3点(在所有其他批处理作业之后)完成)。
If you truly don't want the target system to be updated until midnight, e.g., just queue the messages up until midnight, and then publish them to the destination system's endpoint.
如果您确实不希望目标系统在午夜之前更新,例如,只需将消息排队到午夜,然后将它们发布到目标系统的端点。
Greg Young has blogged and presented extensively on this kind of architecture. Check out his work on InfoQ.
Greg Young在博客上发表了大量有关此类架构的文章。看看他在InfoQ上的工作。