windows server 2003 sp2
apache2.2
mod_jk1.2.26
tomcat6.0.13解压版
sqlserver2005+sp4
现象说明
该项目目前运行在生产环境上，经常出现运行几天之后，tomcat抛heap内存溢出异常,jk无法转发请求给tocmat.
然后一段时间后，通过端口号还可以直接访问tomcat，但是jk还是无法转发。必须把apache和tomcat两个都重启之后，才能恢复。国家单位生产环境的重启真的很蛋疼！
运行情况
平时同时在线人数大概在200上下，因为项目里有查询报表的功能，现在报表的功能实现比较笨，从数据库捞一批数据出来，再在java代码里计算。这里会占用比较多的jvm heap 内存。
疑问
1.什么原因会导致tomcat在内存溢出恢复后，JK还是无法进行转发？如何配置能解决这一个情况？
2.因为是32位的系统,一个jvm只有1.5G的总内存。服务器总的内存有12G，前段时间还特意将几个工程拆开放在3个tomcat中，主功能独立一个tomcat，但还是报内存溢出的问题。有没办法从配置上去解决呢，或者调整GC回收类型是否有助于改善？因为有时候通过tomcat的管理端进去，看到jvm内存就浮动很厉害，有时候1000多M可用，有时候立马降到100M。
配置文件
workers.properties
workers.java_home=D:/install/jdk1.6.0_06
ps=/
worker.list=ajp13,tomcat2,tomcat3,status
worker.ajp13.port=8009
worker.ajp13.host=127.0.0.1
worker.ajp13.type=ajp13
worker.tomcat2.port=9009
worker.tomcat2.host=127.0.0.1
worker.tomcat2.type=ajp13
worker.tomcat3.port=7009
worker.tomcat3.host=127.0.0.1
worker.tomcat3.type=ajp13
worder.status.type=status
catalina.bat
set JAVA_OPTS=-server -Xms1200M -Xmx1200M -Xss512k -XX:PermSize=256M -XX:MaxPermSize=300M -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:LargePageSizeInBytes=256m -XX:+UseFastAccessorMethods -Djava.awt.headless=true
httpd-mpm.conf
# WinNT MPM
# ThreadsPerChild: constant number of worker threads in the server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_winnt_module>
ThreadsPerChild 250
ThreadLimit 300
MaxRequestsPerChild 50000
Win32DisableAcceptEx
EnableMMAP Off
EnableSendfile Off
</IfModule>
server.xml
<!-- Define an AJP 1.3 Connector on port 9009 -->
<Connector port="9009" protocol="AJP/1.3" redirectPort="9443"
maxHttpHeaderSize="8192" maxThreads="250" minSpareThreads="200" maxSpareThreads="250"
enableLookups="false" acceptCount="250" connectionTimeout="30000" disableUploadTimeout="true"
useURIValidationHack="false"
compression="on" compressionMinSize="2048"
compressableMimeType="text/html,text/xml,text/javascript,text/css,text/plain"/>
tomcat异常：
2013-11-8 8:59:14 org.apache.jk.common.ChannelSocket processConnection
警告: processCallbacks status 2
2013-11-8 8:59:14 org.apache.jk.common.ChannelSocket processConnection
警告: processCallbacks status 2
2013-11-8 8:59:21 org.apache.jk.common.ChannelSocket processConnection
警告: processCallbacks status 2
2013-11-8 8:59:49 org.apache.jk.core.MsgContext action
警告: Error sending end packet
java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:531)
at org.apache.jk.common.JkInputStream.endMessage(JkInputStream.java:121)
at org.apache.jk.core.MsgContext.action(MsgContext.java:301)
at org.apache.coyote.Response.action(Response.java:183)
at org.apache.coyote.Response.finish(Response.java:305)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:195)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:686)
at java.lang.Thread.run(Thread.java:619)
2013-11-8 9:00:09 org.apache.jk.common.ChannelSocket processConnection
警告: processCallbacks status 2
2013-11-8 9:05:34 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
严重: Caught exception (java.lang.OutOfMemoryError: Java heap space) executing org.apache.jk.common.ChannelSocket$SocketAcceptor@163305a, terminating thread
2013-11-8 9:09:33 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
严重: Caught exception (java.lang.OutOfMemoryError: Java heap space) executing org.apache.jk.common.ChannelSocket$SocketConnection@1dd8c60, terminating thread
2013-11-8 9:10:00 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
严重: Caught exception (java.lang.OutOfMemoryError: Java heap space) executing org.apache.jk.common.ChannelSocket$SocketConnection@4da615, terminating thread
2013-11-8 9:10:00 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
严重: Caught exception (java.lang.OutOfMemoryError: Java heap space) executing org.apache.jk.common.ChannelSocket$SocketConnection@1a6ac60, terminating thread
2013-11-8 9:10:00 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
严重: Caught exception (java.lang.OutOfMemoryError: Java heap space) executing org.apache.jk.common.ChannelSocket$SocketConnection@12fa165, terminating thread
2013-11-8 9:18:05 org.apache.jasper.runtime.JspFactoryImpl internalGetPageContext
严重: Exception initializing page context
java.lang.OutOfMemoryError: Java heap space
2013-11-8 9:18:11 org.apache.jasper.runtime.JspFactoryImpl internalGetPageContext
严重: Exception initializing page context
java.lang.OutOfMemoryError: Java heap space
2013-11-8 9:32:34 org.apache.jk.core.MsgContext action
警告: Error sending end packet
JK异常日志
[code=java][Wed Aug 28 09:04:02.752 2013] [4744:4500] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:04:02.752 2013] [4744:4500] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:04:02.767 2013] [4744:4500] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:04:34.111 2013] [4744:6312] [info] jk_ajp_common.c (1347): (ajp13) all endpoints are disconnected, detected by connect check (3), cping (0), send (0)
[Wed Aug 28 09:05:00.720 2013] [4744:2112] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:05:00.720 2013] [4744:2112] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:05:00.720 2013] [4744:2112] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:05:33.892 2013] [4744:7780] [info] jk_ajp_common.c (1347): (ajp13) all endpoints are disconnected, detected by connect check (3), cping (0), send (0)
[Wed Aug 28 09:05:39.564 2013] [4744:6316] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:05:39.564 2013] [4744:6316] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:05:39.564 2013] [4744:6316] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:05:42.517 2013] [4744:2744] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:05:42.517 2013] [4744:2744] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:05:42.517 2013] [4744:2744] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:05:59.627 2013] [4744:4556] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:05:59.627 2013] [4744:4556] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:05:59.627 2013] [4744:4556] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:06:29.502 2013] [4744:7516] [error] jk_ajp_common.c (1011): (ajp13) can't receive the response message from tomcat, network problems or tomcat (127.0.0.1:8009) is down (errno=53)
[Wed Aug 28 09:06:29.502 2013] [4744:7516] [error] jk_ajp_common.c (1796): (ajp13) Tomcat is down or network problems. Part of the response has already been sent to the client
[Wed Aug 28 09:06:29.502 2013] [4744:7516] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (recoverable), (attempt=1)
[Wed Aug 28 09:07:11.814 2013] [4744:3440] [info] jk_ajp_common.c (1603): Writing to client aborted or client network problems
[Wed Aug 28 09:07:11.814 2013] [4744:3440] [info] jk_ajp_common.c (2186): (ajp13) sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Wed Aug 28 09:07:11.814 2013] [4744:3440] [info] mod_jk.c (2357): Aborting connection for worker=ajp13
[Wed Aug 28 09:07:54.377 2013] [4744:6640] [info] jk_ajp_common.c (1347): (ajp13) all endpoints are disconnected, detected by connect check (16), cping (0), send (0)
3 个解决方案
#1
还是找找溢出的原因吧。
从根本上解决问题。
从根本上解决问题。
#2
tomcat内存溢出的问题还是要根据你的业务来，你的jvm总内存才1.5G，那么你每次query并发生成报表消耗的内存大概多少，如果超过了这个，那么只能业务上解决了。
#3
改代码是最佳方法。
#1
还是找找溢出的原因吧。
从根本上解决问题。
从根本上解决问题。
#2
tomcat内存溢出的问题还是要根据你的业务来，你的jvm总内存才1.5G，那么你每次query并发生成报表消耗的内存大概多少，如果超过了这个，那么只能业务上解决了。
#3
改代码是最佳方法。