This simple sample code demonstrates the problem. I create an ArrayBlockingQueue, and a thread that waits for data on this queue using take(). After the loop is over, in theory both the queue and the thread can be garbage collected, but in practice I soon get an OutOfMemoryError. What is preventing this to be GC'd, and how can this be fixed?
这个简单的示例代码演示了该问题。我创建了一个ArrayBlockingQueue,以及一个使用take()等待此队列上的数据的线程。循环结束后,理论上队列和线程都可以被垃圾收集,但实际上我很快就会得到一个OutOfMemoryError。是什么阻止了这个GC,以及如何解决这个问题?
/**
* Produces out of memory exception because the thread cannot be garbage
* collected.
*/
@Test
public void checkLeak() {
int count = 0;
while (true) {
// just a simple demo, not useful code.
final ArrayBlockingQueue<Integer> abq = new ArrayBlockingQueue<Integer>(2);
final Thread t = new Thread(new Runnable() {
@Override
public void run() {
try {
abq.take();
} catch (final InterruptedException e) {
e.printStackTrace();
}
}
});
t.start();
// perform a GC once in a while
if (++count % 1000 == 0) {
System.out.println("gc");
// this should remove all the previously created queues and threads
// but it does not
System.gc();
}
}
}
I am using Java 1.6.0.
我使用的是Java 1.6.0。
UPDATE: perform GC after a few iterations, but this does not help.
更新:几次迭代后执行GC,但这没有帮助。
7 个解决方案
#1
Threads are top level objects. They are 'special' so they do not follow the same rules as other objects. The do not rely on references to keep them 'alive' (i.e. safe from GC). A thread will not get garbage collected until it has ended. Which doesn't happen in your sample code, since the thread is blocked. Of course, now that the thread object is not garbage collected, then any other object referenced by it (the queue in your case) also cannot be garbage collected.
线程是*对象。它们是“特殊的”,因此它们不遵循与其他对象相同的规则。不依赖于引用来保持它们“活着”(即从GC中安全)。线程在结束之前不会收集垃圾。由于线程被阻止,因此您的示例代码中不会发生这种情况。当然,既然线程对象没有被垃圾收集,那么它引用的任何其他对象(在你的情况下是队列)也不能被垃圾收集。
#2
You are creating threads indefinitely because they all block until ArrayBlockingQueue<Integer> abq has some entry. So eventually you'll get a OutOfMemoryError.
您正在无限期地创建线程,因为它们都会阻塞,直到ArrayBlockingQueue
(edit)
Each thread you create will never end because it blocks until the abq queue as one entry. If the thread is running, the GC isn't going to collect any object that the thread is referencing including the queue abq and the thread itself.
您创建的每个线程永远不会结束,因为它会阻塞直到abq队列为一个条目。如果线程正在运行,则GC不会收集线程引用的任何对象,包括队列abq和线程本身。
#3
abq.put(0);
should save your day.
应该节省你的一天。
Your threads all wait on their queue's take() but you never put anything in those queues.
你的线程都在等待队列的take(),但是你从不在这些队列中放置任何东西。
#4
Your while loop is an infinite loop and its creating new threads continuously. Although you starting the thread execution as soon as its created but the time its taking to complete the task by the thread is greater then the time its taking to create the thread.
你的while循环是一个无限循环,它不断创建新的线程。虽然你一旦创建了线程执行就开始了,但是线程完成任务所花费的时间比创建线程的时间要长。
Also what are doing with the abq parameter by declaring it inside the while loop?
通过在while循环中声明它,使用abq参数做了什么?
Based on your edits and other comments. System.gc() doesn't not guarantee a GC cycle. Read my statement above the speed of execution of your thread is lower than the speed of creation.
根据您的编辑和其他评论。 System.gc()不保证GC循环。阅读上面的声明,你的线程执行速度低于创建速度。
I checked the comment for the take() method "Retrieves and removes the head of this queue, waiting if no elements are present on this queue." I see you define the ArrayBlockingQueue but you not adding any elements to it so all your thread are just waiting on that method, that is why you getting OOM.
我检查了take()方法的注释“检索并删除此队列的头部,等待此队列中没有元素。”我看到你定义了ArrayBlockingQueue,但你没有添加任何元素,所以你的所有线程都在等待那个方法,这就是你获得OOM的原因。
#5
I do not know how threads are implemented in Java, but one possible reason comes to mind why the queues and threads are not collected: The threads may be wrappers for system threads using system synchronization primitives, in which case the GC cannot automatically collect a waiting thread, since it cannot tell whether the thread is alive or not, i.e. the GC simply does not know that a thread cannot be woken.
我不知道如何在Java中实现线程,但是有一个可能的原因让人想到为什么不收集队列和线程:线程可能是使用系统同步原语的系统线程的包装器,在这种情况下GC不能自动收集等待线程,因为它无法判断线程是否存活,即GC根本不知道线程不能被唤醒。
I can't say what's the best way to fix it, since I'd need to know what you are trying to do, but you could look at java.util.concurrent to see if it has classes for doing what you need.
我不知道解决它的最佳方法是什么,因为我需要知道你要做什么,但是你可以查看java.util.concurrent以查看它是否有用于执行所需操作的类。
#6
You start the thread, so all those new threads will be running asynchronously while the loop continues to create new ones.
您启动该线程,因此所有这些新线程将以异步方式运行,同时循环继续创建新线程。
Since your code is locking, the threads are life references in the system and cannot be collected. But even if they were doing some work, the threads are unlikely to be terminating as quickly as they are created (at least in this sample), and therefore the GC cannot collect all memory and will eventually fail with an OutOfMemoryException.
由于您的代码是锁定的,因此线程是系统中的生命引用,无法收集。但即使他们正在做一些工作,线程也不可能像创建它们那样快地终止(至少在本示例中),因此GC无法收集所有内存,最终会因OutOfMemoryException而失败。
Creating as many threads is neither efficient nor efficient. If it is not a requirement to have all those pending operations run in parallel, you may want to use a thread pool and a queue of runnables to process.
创建尽可能多的线程既不高效也不高效。如果不要求并行运行所有这些挂起操作,则可能需要使用线程池和runnables队列来处理。
#7
The System.gc call does nothing because there is nothing to collect. When a thread starts it increments the threads reference count, not doing so will mean the thread will terminate indeterminately. When the thread's run method completes, then the thread's reference count is decremented.
System.gc调用什么都不做,因为没有什么可以收集。当线程启动时,它会增加线程引用计数,但这样做意味着线程将不确定地终止。当线程的run方法完成时,线程的引用计数递减。
while (true) {
// just a simple demo, not useful code.
// 0 0 - the first number is thread reference count, the second is abq ref count
final ArrayBlockingQueue<Integer> abq = new ArrayBlockingQueue<Integer>(2);
// 0 1
final Thread t = new Thread(new Runnable() {
@Override
public void run() {
try {
abq.take();
// 2 2
} catch (final InterruptedException e) {
e.printStackTrace();
}
}
});
// 1 1
t.start();
// 2 2 (because the run calls abq.take)
// after end of loop
// 1 1 - each created object's reference count is decreased
}
Now, there is a potential race condition - what if the main loop terminates and does garbage collection before the thread t has a chance to do any processing, i.e. it is suspended by the OS before the abq.take statement is executed? The run method will try to access the abq object after the GC has released it, which would be bad.
现在,有一个潜在的竞争条件 - 如果主循环终止并且在线程t有机会进行任何处理之前进行垃圾收集,即在执行abq.take语句之前它被OS挂起,该怎么办?运行方法将尝试在GC释放后访问abq对象,这将是不好的。
To avoid the race condition, you should pass the object as a parameter to the run method. I'm not sure about Java these days, it's been a while, so I'd suggest passing the object as a constructor parameter to a class derived from Runnable. That way, there's an extra reference to abq made before the run method is called, thus ensuring the object is always valid.
要避免竞争条件,应将对象作为参数传递给run方法。我现在不确定Java,已经有一段时间了,所以我建议将对象作为构造函数参数传递给从Runnable派生的类。这样,在调用run方法之前有一个额外的abq引用,从而确保对象始终有效。
#1
Threads are top level objects. They are 'special' so they do not follow the same rules as other objects. The do not rely on references to keep them 'alive' (i.e. safe from GC). A thread will not get garbage collected until it has ended. Which doesn't happen in your sample code, since the thread is blocked. Of course, now that the thread object is not garbage collected, then any other object referenced by it (the queue in your case) also cannot be garbage collected.
线程是*对象。它们是“特殊的”,因此它们不遵循与其他对象相同的规则。不依赖于引用来保持它们“活着”(即从GC中安全)。线程在结束之前不会收集垃圾。由于线程被阻止,因此您的示例代码中不会发生这种情况。当然,既然线程对象没有被垃圾收集,那么它引用的任何其他对象(在你的情况下是队列)也不能被垃圾收集。
#2
You are creating threads indefinitely because they all block until ArrayBlockingQueue<Integer> abq has some entry. So eventually you'll get a OutOfMemoryError.
您正在无限期地创建线程,因为它们都会阻塞,直到ArrayBlockingQueue
(edit)
Each thread you create will never end because it blocks until the abq queue as one entry. If the thread is running, the GC isn't going to collect any object that the thread is referencing including the queue abq and the thread itself.
您创建的每个线程永远不会结束,因为它会阻塞直到abq队列为一个条目。如果线程正在运行,则GC不会收集线程引用的任何对象,包括队列abq和线程本身。
#3
abq.put(0);
should save your day.
应该节省你的一天。
Your threads all wait on their queue's take() but you never put anything in those queues.
你的线程都在等待队列的take(),但是你从不在这些队列中放置任何东西。
#4
Your while loop is an infinite loop and its creating new threads continuously. Although you starting the thread execution as soon as its created but the time its taking to complete the task by the thread is greater then the time its taking to create the thread.
你的while循环是一个无限循环,它不断创建新的线程。虽然你一旦创建了线程执行就开始了,但是线程完成任务所花费的时间比创建线程的时间要长。
Also what are doing with the abq parameter by declaring it inside the while loop?
通过在while循环中声明它,使用abq参数做了什么?
Based on your edits and other comments. System.gc() doesn't not guarantee a GC cycle. Read my statement above the speed of execution of your thread is lower than the speed of creation.
根据您的编辑和其他评论。 System.gc()不保证GC循环。阅读上面的声明,你的线程执行速度低于创建速度。
I checked the comment for the take() method "Retrieves and removes the head of this queue, waiting if no elements are present on this queue." I see you define the ArrayBlockingQueue but you not adding any elements to it so all your thread are just waiting on that method, that is why you getting OOM.
我检查了take()方法的注释“检索并删除此队列的头部,等待此队列中没有元素。”我看到你定义了ArrayBlockingQueue,但你没有添加任何元素,所以你的所有线程都在等待那个方法,这就是你获得OOM的原因。
#5
I do not know how threads are implemented in Java, but one possible reason comes to mind why the queues and threads are not collected: The threads may be wrappers for system threads using system synchronization primitives, in which case the GC cannot automatically collect a waiting thread, since it cannot tell whether the thread is alive or not, i.e. the GC simply does not know that a thread cannot be woken.
我不知道如何在Java中实现线程,但是有一个可能的原因让人想到为什么不收集队列和线程:线程可能是使用系统同步原语的系统线程的包装器,在这种情况下GC不能自动收集等待线程,因为它无法判断线程是否存活,即GC根本不知道线程不能被唤醒。
I can't say what's the best way to fix it, since I'd need to know what you are trying to do, but you could look at java.util.concurrent to see if it has classes for doing what you need.
我不知道解决它的最佳方法是什么,因为我需要知道你要做什么,但是你可以查看java.util.concurrent以查看它是否有用于执行所需操作的类。
#6
You start the thread, so all those new threads will be running asynchronously while the loop continues to create new ones.
您启动该线程,因此所有这些新线程将以异步方式运行,同时循环继续创建新线程。
Since your code is locking, the threads are life references in the system and cannot be collected. But even if they were doing some work, the threads are unlikely to be terminating as quickly as they are created (at least in this sample), and therefore the GC cannot collect all memory and will eventually fail with an OutOfMemoryException.
由于您的代码是锁定的,因此线程是系统中的生命引用,无法收集。但即使他们正在做一些工作,线程也不可能像创建它们那样快地终止(至少在本示例中),因此GC无法收集所有内存,最终会因OutOfMemoryException而失败。
Creating as many threads is neither efficient nor efficient. If it is not a requirement to have all those pending operations run in parallel, you may want to use a thread pool and a queue of runnables to process.
创建尽可能多的线程既不高效也不高效。如果不要求并行运行所有这些挂起操作,则可能需要使用线程池和runnables队列来处理。
#7
The System.gc call does nothing because there is nothing to collect. When a thread starts it increments the threads reference count, not doing so will mean the thread will terminate indeterminately. When the thread's run method completes, then the thread's reference count is decremented.
System.gc调用什么都不做,因为没有什么可以收集。当线程启动时,它会增加线程引用计数,但这样做意味着线程将不确定地终止。当线程的run方法完成时,线程的引用计数递减。
while (true) {
// just a simple demo, not useful code.
// 0 0 - the first number is thread reference count, the second is abq ref count
final ArrayBlockingQueue<Integer> abq = new ArrayBlockingQueue<Integer>(2);
// 0 1
final Thread t = new Thread(new Runnable() {
@Override
public void run() {
try {
abq.take();
// 2 2
} catch (final InterruptedException e) {
e.printStackTrace();
}
}
});
// 1 1
t.start();
// 2 2 (because the run calls abq.take)
// after end of loop
// 1 1 - each created object's reference count is decreased
}
Now, there is a potential race condition - what if the main loop terminates and does garbage collection before the thread t has a chance to do any processing, i.e. it is suspended by the OS before the abq.take statement is executed? The run method will try to access the abq object after the GC has released it, which would be bad.
现在,有一个潜在的竞争条件 - 如果主循环终止并且在线程t有机会进行任何处理之前进行垃圾收集,即在执行abq.take语句之前它被OS挂起,该怎么办?运行方法将尝试在GC释放后访问abq对象,这将是不好的。
To avoid the race condition, you should pass the object as a parameter to the run method. I'm not sure about Java these days, it's been a while, so I'd suggest passing the object as a constructor parameter to a class derived from Runnable. That way, there's an extra reference to abq made before the run method is called, thus ensuring the object is always valid.
要避免竞争条件,应将对象作为参数传递给run方法。我现在不确定Java,已经有一段时间了,所以我建议将对象作为构造函数参数传递给从Runnable派生的类。这样,在调用run方法之前有一个额外的abq引用,从而确保对象始终有效。