We have a Websphere Java EE application that requires parallelization, for which we're looking to use CommonJ work components.
我们有一个需要并行化的Websphere Java EE应用程序,我们正在寻求使用CommonJ工作组件。
Each 'thread' would require its own view onto data coming from a database. Most of this would be pre-fetched, but it would still need to go to the database to get some. We anticipate that the duration of the overall work of all of these threads would be 'long' (i.e. enough time for underlying data to be changed).
每个“线程”都需要自己查看来自数据库的数据。其中大部分都是预先提取的,但它仍然需要去数据库才能得到一些。我们预计所有这些线程的整体工作持续时间将“很长”(即,底层数据有足够的时间进行更改)。
As a result we need to ensure isolation of the data in use by the application and queried during the course of the threads' work.
因此,我们需要确保应用程序使用的数据隔离,并在线程工作期间查询。
It seems that the only way to ensure this is to have a 'global' transaction and use XA transactions. But we'd like to avoid this complexity (and overhead) if at all possible, and are looking for ideas or alternatives: any thoughts?
似乎确保这一点的唯一方法是进行“全局”交易并使用XA交易。但是,如果可能的话,我们希望避免这种复杂性(和开销),并且正在寻找想法或替代方案:任何想法?
Also, to what degree (if at all) do Common-J Work Components support container managed transactions?
此外,Common-J Work Components在多大程度上(如果有的话)支持容器管理的事务?
@Karl: Perhaps I just meant overhead. Our thought was that XA transactions and Messaging incur overhead that Common-J work components sharing a transaction could avoid? The dataset being operated on would be >300k distinct rows of data, each needing ~100 calculations done on it. Whereas these could be divided up onto different threads operating on shared, cached, read-only data, the comparative memory overhead of copying onto/reading off a queue seems prohibitive. Would you agree?
@Karl:也许我的意思是开销。我们的想法是,XA事务和消息传递会导致共享事务的Common-J工作组件可以避免的开销?正在操作的数据集将是> 300k个不同的数据行,每个数据行需要进行约100次计算。虽然这些可以划分为在共享,缓存,只读数据上操作的不同线程,但是复制到队列/读取队列的比较内存开销似乎过高。你同意吗?
@Karl: Tens to low-hundreds of milliseconds per entity. We're also focusing on improving the logic processing as a separate task.
Do I need to use XA transactions when what is required that all threads have a consistent view of data in a single database? My answer to this is that each thread would require its own JPA EntityManager (e.g. connection) and would need XA in order to coordinate their access.
But, if I can do this without XA then so much the better, no?
@Karl:每个实体几十毫秒到几百毫秒。我们还专注于将逻辑处理改进为一项单独的任务。当需要所有线程在单个数据库中具有一致的数据视图时,是否需要使用XA事务?我对此的回答是每个线程都需要自己的JPA EntityManager(例如连接),并且需要XA才能协调它们的访问。但是,如果我能在没有XA的情况下做到这一点,那就更好了,不是吗?
1 个解决方案
#1
What do you find complicated about XA transaction in WAS? By the sounds of it XA transaction sounds like a good fit if your worried about data integrity.
你对WAS中的XA交易有什么看法?通过它的声音,如果您担心数据完整性,XA事务听起来很合适。
When it comes to creating and managing your own thread within WAS it can get a little nasty, I would try to keep away from managing your own threads. One possible may to make things concurrent is to publish data to a queue or a topic and have a large number of multiple concurrent listeners receiving off of the queue. This way you can configure the concurrency and let the container manage the threads.
当谈到在WAS中创建和管理自己的线程时,它可能会有点讨厌,我会尽量远离管理自己的线程。并发的一种可能是将数据发布到队列或主题,并且有大量多个并发侦听器从队列中接收。这样您就可以配置并发性并让容器管理线程。
Karl
#1
What do you find complicated about XA transaction in WAS? By the sounds of it XA transaction sounds like a good fit if your worried about data integrity.
你对WAS中的XA交易有什么看法?通过它的声音,如果您担心数据完整性,XA事务听起来很合适。
When it comes to creating and managing your own thread within WAS it can get a little nasty, I would try to keep away from managing your own threads. One possible may to make things concurrent is to publish data to a queue or a topic and have a large number of multiple concurrent listeners receiving off of the queue. This way you can configure the concurrency and let the container manage the threads.
当谈到在WAS中创建和管理自己的线程时,它可能会有点讨厌,我会尽量远离管理自己的线程。并发的一种可能是将数据发布到队列或主题,并且有大量多个并发侦听器从队列中接收。这样您就可以配置并发性并让容器管理线程。
Karl