不多说,直接上干货!
对于在oozie里放置Mysql驱动包的总结
根据网上的参考以及我个人经验安装的使用
(1)放一份到$OOZIE_HOME/libext下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(2)放一份到$OOZIE_HOME/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(3)放一份到/usr/share/java/下(一定要重命名,mv mysql-connector-java-5.1.21-bin.jar mysql-connector-java.jar)(这里放到bigdatamaster节点)
(4)放一份到$JAVA_HOME/jre/lib/ext (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点、bigdataslave1节点和bigdataslave2节点)
(5)放一份到$OOZIE_HOME/oozie-server/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(1)放一份到$OOZIE_HOME/libext下
[hadoop@bigdatamaster libext]$ pwd
/home/hadoop/app/oozie-4.1.-cdh5.5.4/libext
[hadoop@bigdatamaster libext]$ ls
activation-1.1.jar commons-logging-1.1.jar hadoop-mapreduce-client-shuffle-2.6.-cdh5.5.4.jar leveldbjni-all-1.8.jar
apacheds-i18n-2.0.-M15.jar commons-math3-3.1..jar hadoop-yarn-api-2.6.-cdh5.5.4.jar log4j-1.2..jar
apacheds-kerberos-codec-2.0.-M15.jar commons-net-3.1.jar hadoop-yarn-client-2.6.-cdh5.5.4.jar mysql-connector-java-5.1..jar
api-asn1-api-1.0.-M20.jar curator-client-2.7..jar hadoop-yarn-common-2.6.-cdh5.5.4.jar netty-3.6..Final.jar
api-util-1.0.-M20.jar curator-framework-2.7..jar hadoop-yarn-server-common-2.6.-cdh5.5.4.jar netty-all-4.0..Final.jar
avro-1.7.-cdh5.5.4.jar curator-recipes-2.7..jar htrace-core4-4.0.-incubating.jar paranamer-2.3.jar
aws-java-sdk-core-1.10..jar ext-2.2 httpclient-4.2..jar protobuf-java-2.5..jar
aws-java-sdk-kms-1.10..jar ext-2.2.zip httpcore-4.2..jar servlet-api-2.5.jar
aws-java-sdk-s3-1.10..jar gson-2.2..jar jackson-annotations-2.2..jar slf4j-api-1.7..jar
commons-beanutils-1.7..jar guava-11.0..jar jackson-core-2.2..jar slf4j-log4j12-1.7..jar
commons-beanutils-core-1.8..jar hadoop-annotations-2.6.-cdh5.5.4.jar jackson-core-asl-1.8..jar snappy-java-1.0.4.1.jar
commons-cli-1.2.jar hadoop-auth-2.6.-cdh5.5.4.jar jackson-databind-2.2..jar stax-api-1.0-.jar
commons-codec-1.4.jar hadoop-aws-2.6.-cdh5.5.4.jar jackson-jaxrs-1.8..jar xercesImpl-2.10..jar
commons-collections-3.2..jar hadoop-client-2.6.-cdh5.5.4.jar jackson-mapper-asl-1.8..jar xml-apis-1.4..jar
commons-compress-1.4..jar hadoop-common-2.6.-cdh5.5.4.jar jackson-xc-1.8..jar xmlenc-0.52.jar
commons-configuration-1.6.jar hadoop-hdfs-2.6.-cdh5.5.4.jar jaxb-api-2.2..jar xz-1.0.jar
commons-digester-1.8.jar hadoop-mapreduce-client-app-2.6.-cdh5.5.4.jar jersey-client-1.9.jar zookeeper-3.4.-cdh5.5.4.jar
commons-httpclient-3.1.jar hadoop-mapreduce-client-common-2.6.-cdh5.5.4.jar jersey-core-1.9.jar
commons-io-2.4.jar hadoop-mapreduce-client-core-2.6.-cdh5.5.4.jar jetty-util-6.1..cloudera..jar
commons-lang-2.4.jar hadoop-mapreduce-client-jobclient-2.6.-cdh5.5.4.jar jsr305-3.0..jar
[hadoop@bigdatamaster libext]$
(2)放一份到$OOZIE_HOME/lib下
[hadoop@bigdatamaster lib]$ pwd
/home/hadoop/app/oozie-4.1.0-cdh5.5.4/lib
[hadoop@bigdatamaster lib]$ ls
activemq-client-5.10.2.jar commons-codec-1.4.jar guava-11.0.2.jar jackson-mapper-asl-1.8.8.jar slf4j-simple-1.7.5.jar
apacheds-i18n-2.0.0-M15.jar commons-logging-1.1.jar hadoop-auth-2.6.0-cdh5.5.4.jar jline-2.11.jar xercesImpl-2.10.0.jar
apacheds-kerberos-codec-2.0.0-M15.jar curator-client-2.7.1.jar hawtbuf-1.10.jar json-simple-1.1.jar xml-apis-1.4.01.jar
api-asn1-api-1.0.0-M20.jar curator-framework-2.7.1.jar httpclient-4.2.5.jar jsr305-1.3.9.jar zookeeper-3.4.5-cdh5.5.4.jar
api-util-1.0.0-M20.jar geronimo-j2ee-management_1.1_spec-1.0.1.jar httpcore-4.2.5.jar oozie-client-4.1.0-cdh5.5.4.jar
commons-cli-1.2.jar geronimo-jms_1.1_spec-1.1.1.jar jackson-core-asl-1.8.8.jar slf4j-api-1.7.5.jar
[hadoop@bigdatamaster lib]$ rz [hadoop@bigdatamaster lib]$ ls
activemq-client-5.10.2.jar commons-codec-1.4.jar guava-11.0.2.jar jackson-mapper-asl-1.8.8.jar slf4j-api-1.7.5.jar
apacheds-i18n-2.0.0-M15.jar commons-logging-1.1.jar hadoop-auth-2.6.0-cdh5.5.4.jar jline-2.11.jar slf4j-simple-1.7.5.jar
apacheds-kerberos-codec-2.0.0-M15.jar curator-client-2.7.1.jar hawtbuf-1.10.jar json-simple-1.1.jar xercesImpl-2.10.0.jar
api-asn1-api-1.0.0-M20.jar curator-framework-2.7.1.jar httpclient-4.2.5.jar jsr305-1.3.9.jar xml-apis-1.4.01.jar
api-util-1.0.0-M20.jar geronimo-j2ee-management_1.1_spec-1.0.1.jar httpcore-4.2.5.jar mysql-connector-java-5.1.21.jar zookeeper-3.4.5-cdh5.5.4.jar
commons-cli-1.2.jar geronimo-jms_1.1_spec-1.1.1.jar jackson-core-asl-1.8.8.jar oozie-client-4.1.0-cdh5.5.4.jar
[hadoop@bigdatamaster lib]$
[hadoop@bigdatamaster lib]$ chmod 755 mysql-connector-java-5.1.21.jar
[hadoop@bigdatamaster lib]$ ll
total 9908
-rwxr-xr-x 1 hadoop hadoop 1178428 Apr 26 2016 activemq-client-5.10.2.jar
-rwxr-xr-x 1 hadoop hadoop 44925 Apr 26 2016 apacheds-i18n-2.0.0-M15.jar
-rwxr-xr-x 1 hadoop hadoop 691479 Apr 26 2016 apacheds-kerberos-codec-2.0.0-M15.jar
-rwxr-xr-x 1 hadoop hadoop 16560 Apr 26 2016 api-asn1-api-1.0.0-M20.jar
-rwxr-xr-x 1 hadoop hadoop 79912 Apr 26 2016 api-util-1.0.0-M20.jar
-rwxr-xr-x 1 hadoop hadoop 41123 Apr 26 2016 commons-cli-1.2.jar
-rwxr-xr-x 1 hadoop hadoop 58160 Apr 26 2016 commons-codec-1.4.jar
-rwxr-xr-x 1 hadoop hadoop 52915 Apr 26 2016 commons-logging-1.1.jar
-rwxr-xr-x 1 hadoop hadoop 69500 Apr 26 2016 curator-client-2.7.1.jar
-rwxr-xr-x 1 hadoop hadoop 186273 Apr 26 2016 curator-framework-2.7.1.jar
-rwxr-xr-x 1 hadoop hadoop 20220 Apr 26 2016 geronimo-j2ee-management_1.1_spec-1.0.1.jar
-rwxr-xr-x 1 hadoop hadoop 32359 Apr 26 2016 geronimo-jms_1.1_spec-1.1.1.jar
-rwxr-xr-x 1 hadoop hadoop 1648200 Apr 26 2016 guava-11.0.2.jar
-rwxr-xr-x 1 hadoop hadoop 73940 Apr 26 2016 hadoop-auth-2.6.0-cdh5.5.4.jar
-rwxr-xr-x 1 hadoop hadoop 50150 Apr 26 2016 hawtbuf-1.10.jar
-rwxr-xr-x 1 hadoop hadoop 433368 Apr 26 2016 httpclient-4.2.5.jar
-rwxr-xr-x 1 hadoop hadoop 227708 Apr 26 2016 httpcore-4.2.5.jar
-rwxr-xr-x 1 hadoop hadoop 227500 Apr 26 2016 jackson-core-asl-1.8.8.jar
-rwxr-xr-x 1 hadoop hadoop 668564 Apr 26 2016 jackson-mapper-asl-1.8.8.jar
-rwxr-xr-x 1 hadoop hadoop 208781 Apr 26 2016 jline-2.11.jar
-rwxr-xr-x 1 hadoop hadoop 16046 Apr 26 2016 json-simple-1.1.jar
-rwxr-xr-x 1 hadoop hadoop 33015 Apr 26 2016 jsr305-1.3.9.jar
-rwxr-xr-x 1 hadoop hadoop 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar
-rwxr-xr-x 1 hadoop hadoop 229725 Apr 26 2016 oozie-client-4.1.0-cdh5.5.4.jar
-rwxr-xr-x 1 hadoop hadoop 26084 Apr 26 2016 slf4j-api-1.7.5.jar
-rwxr-xr-x 1 hadoop hadoop 10680 Apr 26 2016 slf4j-simple-1.7.5.jar
-rwxr-xr-x 1 hadoop hadoop 1363159 Apr 26 2016 xercesImpl-2.10.0.jar
-rwxr-xr-x 1 hadoop hadoop 220536 Apr 26 2016 xml-apis-1.4.01.jar
-rwxr-xr-x 1 hadoop hadoop 1352025 Apr 26 2016 zookeeper-3.4.5-cdh5.5.4.jar
[hadoop@bigdatamaster lib]$
(3)
这个报错很常见,原以为把mysql-connector-java-*-bin.jar丢到oozie的lib即可,但是发现依然宝盖错误。后来在网上找到需要将这个包放到这个路径下就通过了(名字需要修改下)
/usr/share/java/mysql-connector-java.jar
[hadoop@bigdatamaster java]$ pwd
/usr/share/java
[hadoop@bigdatamaster java]$ su root
Password:
[root@bigdatamaster java]# ls
ant dom3-xerces-j2-2.7.1.jar java_cup-runtime-0.10k.jar js-1.7.jar rhino.jar xml-commons-apis.jar
ant-1.7.1.jar dom3-xerces-j2.jar java_cup-runtime.jar js.jar sinjdoc.jar xml-commons-jaxp-1.3-apis.jar
ant-bootstrap-1.7.1.jar dom3-xml-commons-apis-1.3.04.jar jaxp13.jar libgcj-4.4.4.jar xerces-j2-2.7.1.jar xml-commons-resolver-1.1.jar
ant-bootstrap.jar dom3-xml-commons-apis.jar jaxp.jar libgcj-4.4.7.jar xerces-j2.jar xml-commons-resolver.jar
ant.jar gcj-endorsed jaxp_parser_impl.jar rhino-1.7.jar xml-commons-apis-1.3.04.jar
ant-launcher-1.7.1.jar java_cup-0.10k.jar jline-0.9.94.jar rhino-examples-1.7.jar xml-commons-apis-ext-1.3.04.jar
ant-launcher.jar java_cup.jar jline.jar rhino-examples.jar xml-commons-apis-ext.jar
[root@bigdatamaster java]# rz [root@bigdatamaster java]# ls
ant dom3-xerces-j2-2.7.1.jar java_cup-runtime-0.10k.jar js-1.7.jar rhino-examples.jar xml-commons-apis-ext.jar
ant-1.7.1.jar dom3-xerces-j2.jar java_cup-runtime.jar js.jar rhino.jar xml-commons-apis.jar
ant-bootstrap-1.7.1.jar dom3-xml-commons-apis-1.3.04.jar jaxp13.jar libgcj-4.4.4.jar sinjdoc.jar xml-commons-jaxp-1.3-apis.jar
ant-bootstrap.jar dom3-xml-commons-apis.jar jaxp.jar libgcj-4.4.7.jar xerces-j2-2.7.1.jar xml-commons-resolver-1.1.jar
ant.jar gcj-endorsed jaxp_parser_impl.jar mysql-connector-java-5.1.21.jar xerces-j2.jar xml-commons-resolver.jar
ant-launcher-1.7.1.jar java_cup-0.10k.jar jline-0.9.94.jar rhino-1.7.jar xml-commons-apis-1.3.04.jar
ant-launcher.jar java_cup.jar jline.jar rhino-examples-1.7.jar xml-commons-apis-ext-1.3.04.jar
[root@bigdatamaster java]# ln -s mysql-connector-java-5.1.21.jar mysql-connector-java.jar
[root@bigdatamaster java]# ll
total 15036
drwxr-xr-x 2 root root 4096 May 11 2016 ant
-rw-r--r-- 1 root root 1322826 May 11 2016 ant-1.7.1.jar
-rw-r--r-- 1 root root 18834 May 11 2016 ant-bootstrap-1.7.1.jar
lrwxrwxrwx 1 root root 23 May 5 20:38 ant-bootstrap.jar -> ant-bootstrap-1.7.1.jar
lrwxrwxrwx 1 root root 13 May 5 20:38 ant.jar -> ant-1.7.1.jar
-rw-r--r-- 1 root root 12129 May 11 2016 ant-launcher-1.7.1.jar
lrwxrwxrwx 1 root root 22 May 5 20:38 ant-launcher.jar -> ant-launcher-1.7.1.jar
lrwxrwxrwx 1 root root 19 May 5 20:36 dom3-xerces-j2-2.7.1.jar -> xerces-j2-2.7.1.jar
lrwxrwxrwx 1 root root 24 May 5 20:36 dom3-xerces-j2.jar -> dom3-xerces-j2-2.7.1.jar
lrwxrwxrwx 1 root root 27 May 5 20:36 dom3-xml-commons-apis-1.3.04.jar -> xml-commons-apis-1.3.04.jar
lrwxrwxrwx 1 root root 32 May 5 20:36 dom3-xml-commons-apis.jar -> dom3-xml-commons-apis-1.3.04.jar
drwxr-xr-x 2 root root 4096 Mar 22 00:04 gcj-endorsed
-rw-r--r-- 1 root root 73310 Aug 21 2010 java_cup-0.10k.jar
lrwxrwxrwx 1 root root 18 May 5 20:36 java_cup.jar -> java_cup-0.10k.jar
-rw-r--r-- 1 root root 9053 Aug 21 2010 java_cup-runtime-0.10k.jar
lrwxrwxrwx 1 root root 26 May 5 20:36 java_cup-runtime.jar -> java_cup-runtime-0.10k.jar
lrwxrwxrwx 1 root root 20 May 5 20:36 jaxp13.jar -> xml-commons-apis.jar
lrwxrwxrwx 1 root root 20 May 5 20:36 jaxp.jar -> xml-commons-apis.jar
lrwxrwxrwx 1 root root 34 May 5 20:36 jaxp_parser_impl.jar -> /etc/alternatives/jaxp_parser_impl
-rw-r--r--. 1 root root 84746 Aug 22 2010 jline-0.9.94.jar
lrwxrwxrwx. 1 root root 16 May 4 18:10 jline.jar -> jline-0.9.94.jar
lrwxrwxrwx. 1 root root 13 May 4 18:10 js-1.7.jar -> rhino-1.7.jar
lrwxrwxrwx. 1 root root 10 May 4 18:10 js.jar -> js-1.7.jar
-rw-r--r-- 1 root root 10143547 Mar 22 00:05 libgcj-4.4.4.jar
lrwxrwxrwx 1 root root 16 May 5 20:36 libgcj-4.4.7.jar -> libgcj-4.4.4.jar
-rw-r--r-- 1 root root 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar
lrwxrwxrwx 1 root root 31 May 9 14:30 mysql-connector-java.jar -> mysql-connector-java-5.1.21.jar
-rw-r--r--. 1 root root 807405 Aug 22 2010 rhino-1.7.jar
-rw-r--r--. 1 root root 18260 Aug 22 2010 rhino-examples-1.7.jar
lrwxrwxrwx. 1 root root 22 May 4 18:10 rhino-examples.jar -> rhino-examples-1.7.jar
lrwxrwxrwx. 1 root root 13 May 4 18:10 rhino.jar -> rhino-1.7.jar
-rw-r--r-- 1 root root 414906 Aug 19 2010 sinjdoc.jar
-rw-r--r-- 1 root root 1246482 Sep 30 2014 xerces-j2-2.7.1.jar
lrwxrwxrwx 1 root root 19 May 5 20:36 xerces-j2.jar -> xerces-j2-2.7.1.jar
-rw-r--r-- 1 root root 230522 Aug 19 2010 xml-commons-apis-1.3.04.jar
-rw-r--r-- 1 root root 90958 Aug 19 2010 xml-commons-apis-ext-1.3.04.jar
lrwxrwxrwx 1 root root 31 May 5 20:36 xml-commons-apis-ext.jar -> xml-commons-apis-ext-1.3.04.jar
lrwxrwxrwx 1 root root 27 May 5 20:36 xml-commons-apis.jar -> xml-commons-apis-1.3.04.jar
lrwxrwxrwx 1 root root 20 May 5 20:36 xml-commons-jaxp-1.3-apis.jar -> xml-commons-apis.jar
-rw-r--r-- 1 root root 56968 Aug 21 2010 xml-commons-resolver-1.1.jar
lrwxrwxrwx 1 root root 28 May 5 20:36 xml-commons-resolver.jar -> xml-commons-resolver-1.1.jar
[root@bigdatamaster java]#
[root@bigdatamaster java]# mv mysql-connector-java-5.1.21.jar mysql-connector-java.jar
mv: overwrite `mysql-connector-java.jar'? y
[root@bigdatamaster java]#
(4)因为,我暂时用的是jdk1.7。 即,在$OOZIE_HOME/libext下的mysql-connector-java-6.0.3.jar 替换成 mysql-connector-java-5.1.21.jar 。
(5)放一份到$OOZIE_HOME/oozie-server/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
[hadoop@bigdatamaster lib]$ pwd
/home/hadoop/app/oozie-4.1.-cdh5.5.4/oozie-server/lib
[hadoop@bigdatamaster lib]$ ls
annotations-api.jar catalina-ha.jar catalina-tribes.jar el-api.jar jasper.jar servlet-api.jar tomcat-dbcp.jar tomcat-i18n-fr.jar
catalina-ant.jar catalina.jar ecj-4.3..jar jasper-el.jar jsp-api.jar tomcat-coyote.jar tomcat-i18n-es.jar tomcat-i18n-ja.jar
[hadoop@bigdatamaster lib]$ rz [hadoop@bigdatamaster lib]$ ls
annotations-api.jar catalina-ha.jar catalina-tribes.jar el-api.jar jasper.jar mysql-connector-java-5.1..jar tomcat-coyote.jar tomcat-i18n-es.jar tomcat-i18n-ja.jar
catalina-ant.jar catalina.jar ecj-4.3..jar jasper-el.jar jsp-api.jar servlet-api.jar tomcat-dbcp.jar tomcat-i18n-fr.jar
[hadoop@bigdatamaster lib]$ ll
total
-rwxr-xr-x hadoop hadoop May annotations-api.jar
-rwxr-xr-x hadoop hadoop May catalina-ant.jar
-rwxr-xr-x hadoop hadoop May catalina-ha.jar
-rwxr-xr-x hadoop hadoop May catalina.jar
-rwxr-xr-x hadoop hadoop May catalina-tribes.jar
-rwxr-xr-x hadoop hadoop May ecj-4.3..jar
-rwxr-xr-x hadoop hadoop May el-api.jar
-rwxr-xr-x hadoop hadoop May jasper-el.jar
-rwxr-xr-x hadoop hadoop May jasper.jar
-rwxr-xr-x hadoop hadoop May jsp-api.jar
-rw-r--r-- hadoop hadoop Nov mysql-connector-java-5.1..jar
-rwxr-xr-x hadoop hadoop May servlet-api.jar
-rwxr-xr-x hadoop hadoop May tomcat-coyote.jar
-rwxr-xr-x hadoop hadoop May tomcat-dbcp.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-es.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-fr.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-ja.jar
[hadoop@bigdatamaster lib]$ chmod mysql-connector-java-5.1..jar
[hadoop@bigdatamaster lib]$ ll
total
-rwxr-xr-x hadoop hadoop May annotations-api.jar
-rwxr-xr-x hadoop hadoop May catalina-ant.jar
-rwxr-xr-x hadoop hadoop May catalina-ha.jar
-rwxr-xr-x hadoop hadoop May catalina.jar
-rwxr-xr-x hadoop hadoop May catalina-tribes.jar
-rwxr-xr-x hadoop hadoop May ecj-4.3..jar
-rwxr-xr-x hadoop hadoop May el-api.jar
-rwxr-xr-x hadoop hadoop May jasper-el.jar
-rwxr-xr-x hadoop hadoop May jasper.jar
-rwxr-xr-x hadoop hadoop May jsp-api.jar
-rwxr-xr-x hadoop hadoop Nov mysql-connector-java-5.1..jar
-rwxr-xr-x hadoop hadoop May servlet-api.jar
-rwxr-xr-x hadoop hadoop May tomcat-coyote.jar
-rwxr-xr-x hadoop hadoop May tomcat-dbcp.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-es.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-fr.jar
-rwxr-xr-x hadoop hadoop May tomcat-i18n-ja.jar
[hadoop@bigdatamaster lib]$
然后,再,
[hadoop@bigdatamaster oozie-4.1.-cdh5.5.4]$ bin/oozie-start.sh [hadoop@bigdatamaster oozie-4.1.-cdh5.5.4]$ jps
Bootstrap
NameNode
SecondaryNameNode
Jps
QuorumPeerMain
ResourceManager
[hadoop@bigdatamaster oozie-4.1.-cdh5.5.4]$
然后呢,大家也许还会出现如下问题: