项目适用kettle作为etl工具,源数据库为mysql库,目标库为oracle。在持续的循环调度中,经常发现oracle的数据库连接中断,需要重置。
具体报错信息如下:
INFO 26-12 23:40:13,220 - Kitchen - Logging is at level : Basic logging
INFO 26-12 23:40:13,221 - Kitchen - Start of run.
INFO 26-12 23:40:13,374 - WAN_PT_OLD_SJ_DAY_ADD - Start of job execution
ERROR 26-12 23:42:50,832 - WAN_PT_OLD_SJ_DAY_ADD - A serious error occurred during job execution: org.pentaho.di.core.exception.KettleDatabaseException:
Error occured while trying to connect to the database
INFO 26-12 23:40:13,221 - Kitchen - Start of run.
INFO 26-12 23:40:13,374 - WAN_PT_OLD_SJ_DAY_ADD - Start of job execution
ERROR 26-12 23:42:50,832 - WAN_PT_OLD_SJ_DAY_ADD - A serious error occurred during job execution: org.pentaho.di.core.exception.KettleDatabaseException:
Error occured while trying to connect to the database
Invalid JNDI connection ETL : IO Error: Connection reset
INFO 26-12 23:42:50,837 - Kitchen - Finished!
ERROR 26-12 23:42:50,837 - Kitchen - Finished with errors
INFO 26-12 23:42:50,838 - Kitchen - Start=2013/12/26 23:40:13.221, Stop=2013/12/26 23:42:50.837
INFO 26-12 23:42:50,838 - Kitchen - Processing ended after 2 minutes and 37 seconds (157 seconds total).
将oracle会话超时不设限,并且将机器的防火墙关闭后,问题仍然存在。
网上查阅资料后发现主要问题是oracle 11g 和centos 64的不兼容bug造成的。
可能原因是1:jvm试图展示/tmp文件夹下的内容,但是/tmp文件内内容过多超时;
2:函数nextBytes(byte[]) 在linux上使用/dev/random计数时,导致登录中断从而超时
解决方法,在sh文件中的java 命令行中增加参数:-Djava.security.egd=file:/dev/../dev/urandom
如在kitchen.sh/pan.sh文件的如下修改后问题即可解决:
1.如果显示内存溢出,即可如上图在javamaxmem处增加内存即可,本机设置为10g,一般不超过内存一半即可。
2.如果连接mysql显示时间超时,将mysql数据库的timeout参数调大即可,自己设置如下: