
Weblogic控制台监控发现 环境>>服务器>>你的服务器>>监控>>线程 中活动执行线程竟然是2000多。同一套系统在另一套平台上,并且访问的人不少,也没有超过100。重启此应用,活动进程依然没有变化,真是奇怪。
查看转储线程堆:
==== FULL THREAD DUMP=============== Tue Aug :: Oracle JRockit(R)R28.1.0---1.6.0_20---linux-x86_64 "Main Thread" id=1idx=0x4 tid= prio= alive, waiting, native_blocked -- Waiting for notification on:weblogic/t3/srvr/T3Srvr@0xc70a6538[fat lock] atjrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method) atjava/lang/Object.wait(J)V(Native Method) atjava/lang/Object.wait(Object.java:) atweblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:) ^-- Lock released whilewaiting: weblogic/t3/srvr/T3Srvr@0xc70a6538[fat lock] atweblogic/t3/srvr/T3Srvr.run(T3Srvr.java:) atweblogic/Server.main(Server.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "(Signal Handler)" id=2idx=0x8 tid= prio= alive, native_blocked, daemon "(OC Main Thread)" id=3idx=0xc tid= prio= alive, native_waiting, daemon "(GC Worker Thread 1)" id=?idx=0x10 tid= prio= alive, daemon "(GC Worker Thread 2)"id=? idx=0x14 tid= prio= alive, daemon "(GC Worker Thread 3)"id=? idx=0x18 tid= prio= alive, daemon "(GC Worker Thread 4)"id=? idx=0x1c tid= prio= alive, daemon "(Code Generation Thread1)" id= idx=0x20 tid= prio= alive, native_waiting, daemon "(Code Optimization Thread1)" id= idx=0x24 tid= prio= alive, native_waiting, daemon "(VM Periodic Task)" id=6idx=0x28 tid= prio= alive, native_blocked, daemon "Finalizer" id= idx=0x2ctid= prio= alive, native_waiting, daemon atjrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(NativeMethod) atjrockit/memory/Finalizer.access$(Finalizer.java:) atjrockit/memory/Finalizer$.run(Finalizer.java:) atjava/lang/Thread.run(Thread.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "Reference Handler" id=8idx=0x30 tid= prio= alive, native_waiting, daemon atjava/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(NativeMethod) atjava/lang/ref/Reference.access$(Reference.java:) atjava/lang/ref/Reference$ReferenceHandler.run(Reference.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "(Sensor Event Thread)"id= idx=0x34 tid= prio= alive, native_blocked, daemon "VM JFR Buffer Thread"id= idx=0x38 tid= prio= alive, in native, daemon "Timer-0" id= idx=0x3ctid= prio= alive, waiting, native_blocked, daemon -- Waiting for notification on:java/util/TaskQueue@0xc504e198[fat lock] atjrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method) at java/lang/Object.wait(J)V(NativeMethod) atjava/lang/Object.wait(Object.java:) atjava/util/TimerThread.mainLoop(Timer.java:) ^-- Lock released whilewaiting: java/util/TaskQueue@0xc504e198[fat lock] atjava/util/TimerThread.run(Timer.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "Timer-1" id= idx=0x40 tid=12416prio= alive, waiting, native_blocked, daemon -- Waiting for notification on:java/util/TaskQueue@0xc504e548[fat lock] atjrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method) atjava/lang/Object.wait(J)V(Native Method) atjava/util/TimerThread.mainLoop(Timer.java:) ^-- Lock released whilewaiting: java/util/TaskQueue@0xc504e548[fat lock] atjava/util/TimerThread.run(Timer.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "[ACTIVE] ExecuteThread: '0'for queue: 'weblogic.kernel.Default (self-tuning)'" id= idx=0x44tid= prio= alive, waiting, native_blocked, daemon -- Waiting for notification on:weblogic/work/ExecuteThread@0xc6e748b0[fat lock] at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(NativeMethod) atjava/lang/Object.wait(J)V(Native Method) atjava/lang/Object.wait(Object.java:) atweblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:) ^-- Lock released whilewaiting: weblogic/work/ExecuteThread@0xc6e748b0[fat lock] atweblogic/work/ExecuteThread.run(ExecuteThread.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "[ACTIVE] ExecuteThread: '1'for queue: 'weblogic.kernel.Default (self-tuning)'" id= idx=0x48tid= prio= alive, waiting, native_blocked, daemon -- Waiting for notification on:weblogic/work/ExecuteThread@0xc6d42d00[fat lock] atjrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method) atjava/lang/Object.wait(J)V(Native Method) atjava/lang/Object.wait(Object.java:) atweblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:) ^-- Lock released while waiting:weblogic/work/ExecuteThread@0xc6d42d00[fat lock] atweblogic/work/ExecuteThread.run(ExecuteThread.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace "[ACTIVE] ExecuteThread: '2'for queue: 'weblogic.kernel.Default (self-tuning)'" id= idx=0x4ctid= prio= alive, waiting, native_blocked, daemon -- Waiting for notification on:weblogic/work/ExecuteThread@0xc6e1c620[fat lock] atjrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method) atjava/lang/Object.wait(J)V(Native Method) atjava/lang/Object.wait(Object.java:) atweblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:) ^-- Lock released whilewaiting: weblogic/work/ExecuteThread@0xc6e1c620[fat lock] atweblogic/work/ExecuteThread.run(ExecuteThread.java:) atjrockit/vm/RNI.c2java(JJJJJ)V(Native Method) -- end of trace
…………此处省略大量重复的信息
在下对weblogic了解甚少,对java开发也是丈二和尚,身边也没有懂weblogic的人,看来只能借助网络搜索了。百度、ITpub、****全试了都没有相关问题。后来加了几个weblogic的qq群,在群里发问基本也是没人管。哎,这世界怎么就没有个好心人呢?不过好心人最终还是出现了,他直接就找到了问题所在——这个问题的原因就在启动参数上。(就是啊,重新启动都不行,那十有八九是启动参数的问题啊,我这笨脑子!)
造成这种情况的参数在weblogic\user_projects\domains\base_domain\bin下的setDomainEnvNaNd中的Dweblogic.threadpool.MinPoolSize。我的配置是这样的:
JAVA_OPTIONS="${JAVA_OPTIONS}${JAVA_PROPERTIES} -Dwlw.iterativeDev=${iterativeDevFlag}-Dwlw.testConsole=${testConsoleFlag} -Dweblogic.threadpool.MinPoolSize=2000 -Dweblogic.threadpool.MaxPoolSize=4000-Dwlw.logErrorsToConsole=${logErrorsToConsoleFlag}"
exportJAVA_OPTIONS
这里的-Dweblogic.threadpool.MinPoolSize=2000意思是默认线程池大小,这个参数设置多少合适我也不知道,具体设置于不设置有什么大的区别也不知道……额,反正是它在搞鬼。我将此参数去掉之后,重新启动,活动线程变少了。
网抄一些weblogic的知识,留作备查——————
检查线程数
通过weblogic控制台可以查看线程数的统计信息。weblogic9及以上的线程是自优化的。但应该查看系统的线程最大数是否过大,如果过大,就要注意系统为什么会有这么大的压力。如下为示例截图
对应中文翻译:
ActiveExecute Threads:在活动的线程池内处理请求的线程个数
ExecuteThread Total Count:线程池内线程的总数
ExecuteThread Idle Count:池内的空闲线程数。它不包含stuck和standby的线程数。它是指等待接收新请求到来并处理的线程个数
queuelength :在等待队列里的请求数,通常保留默认值 65536 ,队列长度表明了同时发来请求的最大数, 65536 个请求是个很大的数,即使达到这个最大数,也是很少见的。如果达到最大队列长度,WebLogic 会自动成倍增长队列大小,以处理额外的工作。 注意:超过 65536 个请求预示队列中的线程有问题,不仅仅只是队列本身的长度问题,实践表明在队列中有堵塞线程或线程数不足的情况存在。
hoggingthread count :线程处理一个请求时间超过一定值被视为hogging状态,如果继续处理请求超过一定时间将被视为stuck,或处理完请求后被放回线程池
standbythread count :统计在standby(备用)线程池内的线程数。这些线程不需要处理当前请求被放入standby池内,当活动的线程池内需要更多线程时,这些线程将被激活。
Execute Thread Total Count= Active Execute Threads+ standbythread count