如何使用ExecutorService修复程序？

So, I'm new in Java.

所以,我是Java新手。

I wrote relatively simple program that does something with a lot of files.

我编写了相对简单的程序,它可以处理很多文件。

It was slow, I wanted to run more threads than one. With little * Community help I made something like this:

它很慢,我想运行更多的线程而不是一个。有了*社区的帮助,我做了类似这样的事情:

public class FileProcessor {
    public static void main(String[] args)
    {
        // run 5 threads
        ExecutorService executor = Executors.newFixedThreadPool(5);
        int i;

        // get first and last file ID to process
        int start = Integer.parseInt(args[0]);
        int end = Integer.parseInt(args[1]);

        for (i = start; i < end; i++)
        {
            final int finalId = i; // final necessary in anonymous class
            executor.submit(new Runnable() 
            {
                public void run() 
                {
                    processFile(finalId);
                }
            });
        }
    }

    public static void processFile(int id)
    {
        //doing work here
    }
}

This is really really simple multithreading solution and it does what I want. Now I want to fix/improve it because I guess I'm doing it wrong (program never ends, uses more memory than it should etc.).

这是非常简单的多线程解决方案,它可以满足我的需求。现在我想修复/改进它,因为我猜我做错了(程序永远不会结束,使用的内存比它应该的多等)。

Shall I reduce number of Runnable objects existing in memory at the same time? If I should - how can I do it?

我应该同时减少内存中存在的Runnable对象的数量吗?如果我应该 - 我怎么能这样做?
How can I detect, that all job is done and exit program (and threads)?

我怎样才能检测到所有工作都已完成并退出程序(和线程)?

2 个解决方案

#1

when you said program never ends, uses more memory than it should, it could be due to many reasons like,

当你说程序永远不会结束时,使用的内存比它应该多,可能是由于很多原因,比如

1) processFile() might be doing some heavy I/O operations (or) it is blocked for some I/O data.

1)processFile()可能正在执行一些繁重的I / O操作(或)它被阻止某些I / O数据。

2) There could be a potential dead lock if there is any common data object share

2)如果存在任何公共数据对象共享,则可能存在潜在的死锁

Your thread logic itself, pretty straight forward with ThreadPoolExecutor and I believe the problem is with the code in processFile().

你的线程逻辑本身,与ThreadPoolExecutor非常直接,我相信问题在于processFile()中的代码。

Since you initialized the pool with 5, The ThreadPoolExecutor makes sure that there are only 5 active threads doing the work irrespective of how many possible threads you want to create.

由于您使用5初始化池,因此ThreadPoolExecutor确保只有5个活动线程正在执行工作,而不管您要创建多少可能的线程。

So In this case, I would focus more on application logic optimization than thread management.

所以在这种情况下,我会更多地关注应用程序逻辑优化而不是线程管理。

If you are really concerned about how many Runnable objects you want to create? Then That's a trade-off between your application requirement and available resources.

如果您真的关心要创建多少个Runnable对象?然后,这是您的应用程序需求和可用资源之间的权衡。

If each thread task is independent and there is a execution time limit for all of those threads, then you create more threads in pool and add more resources.

如果每个线程任务都是独立的,并且所有这些线程都有执行时间限制,那么您在池中创建更多线程并添加更多资源。

When you define a PoolExecutor with 5 threads at a time limit, and create 10,000 threads then obviously they have to wait as a Future Task in memory until a thread is available.

当您在一个时间限制上定义一个具有5个线程的PoolExecutor并创建10,000个线程时,显然它们必须在内存中等待Future Task,直到线程可用。

#2

If you want to reduce the number of threads running, just reduce the size you pass to the fixed thread pool constructor. As for termination, call shutdown and awaitTermination on the executor service. But that will only reduce the number of active threads, not the number of Runnables you're creating in your loop.

如果要减少运行的线程数,只需减小传递给固定线程池构造函数的大小。至于终止,在执行程序服务上调用shutdown和awaitTermination。但这只会减少活动线程的数量,而不会减少您在循环中创建的Runnable的数量。

#1