一、maven安装
1) 从网站上下载Linux版本的Maven安装包;
apache-maven-3.0.5-bin.tar.gz下载
具体解压目录在 /home/maven/
========================================
2) 解压apache-maven-3.0.5-bin.tar.gz进行安装;
tar -xzvf apache-maven-3.0.5-bin.tar.gz
3) 设置maven环境变量;
vim /etc/profile或者vim /etc/environment
在末尾处增加如下内容:
#maven envirment
export M2_HOME=/home/maven
export M2=$M2_HOME/bin
export MAVEN_OPTS="-Xms256m -Xmx512m"
export PATH=$M2:$PATH
4) 使配置文件生效
source /etc/profile
5)验证maven是否安装成功。
root@dm4:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin# mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 21:51:28+0800)
Maven home: /home/maven
Java version: 1.6.0_43, vendor: Sun Microsystems Inc.
Java home: /usr/lib/jvm/jdk1.6.0_43/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-24-server", arch: "amd64", family: "unix"
二、oozie
1、Build
bin/mkdistro.sh -DskipTests
成功build后,会在目录里生成一个distribution.
hadoop安装时创建的hadoop组和hadoop用户对oozie目录是没有权限,所以最好一开始创建oozie组和oozie用户,同时赋予权限,不然会报错.偷懒用了root用户。
PS:后来发现,可以单独拷贝/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2下的oozie-3.3.2-distro.tar.gz包到用户自定义目录进行解压使用,除此之外的所有文件可以全部删除。
2、安装
根据官方文档在distibution的目录下(/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2)创建libext,因为装的hadoop版本是1.0.3,所以拷贝$HADOOP_HOME/lib下的所有jar包到libext下,同时拷贝ext-2.2.zip到该目录下。
还要拷贝$HADOOP_HOME下的所有jar包到libext.
因为oozie默认并不支持1.0.3版,所以对/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin文件夹下的addtowar.sh文件进行修改:
elif [ "${version}" = "0.20.200" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar:commons-configuration-*.jar"
后边增加如下内容:elif [ "${version}" = "1.0.3" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar:commons-configuration-*.jar"
运行oozie-setup.sh创建oozie.war
root@dm4:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2# bin/oozie-setup.sh -hadoop 1.0.3 /home/hadoop -extjs /home/oozie/ext-2.2.zip
执行成功后会出现如下界面:
拷贝oozie.war到tomcat部署目录下
root@ubuntu:/home/oozie/webapp/src/main/webapp# cp /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie.war ./
修改配置文件:
<property>
<name>oozie.service.JPAService.create.db.schema</name>
<value>true</value>
<description>
Creates Oozie DB.
If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP.
If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up.
</description>
</property>
创建Oozie DB
cd /usr/local/oozie-3.3.1/distro/target/oozie-3.3.1-distro/oozie-3.3.1/bin
./ooziedb.sh create -sqlfile oozie.sql -run
结果
DONE
Check DB schema does not exist
DONE
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
DONE
Create OOZIE_SYS table
DONE
Oozie DB has been created for Oozie version '3.3.1'
运行oozie
cd /usr/local/oozie-3.3.1/distro/target/oozie-3.3.1-distro/oozie-3.3.1/bin
./oozie-start.sh
开启服务:
bin/oozie-run.sh
运行结果
root@dm4:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2# bin/oozie-run.sh
WARN: Use of this script is deprecated; use 'oozied.sh run' instead
Setting OOZIE_HOME: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2
Setting OOZIE_CONFIG: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/conf
Sourcing: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/conf/oozie-env.sh
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
Setting OOZIE_CONFIG_FILE: oozie-site.xml
Setting OOZIE_DATA: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/data
Setting OOZIE_LOG: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/logs
Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD: 10
Setting OOZIE_HTTP_HOSTNAME: dm4
Setting OOZIE_HTTP_PORT: 11000
Setting OOZIE_ADMIN_PORT: 11001
Setting OOZIE_HTTPS_PORT: 11443
Setting OOZIE_BASE_URL: http://dm4:11000/oozie
Setting CATALINA_BASE: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server
Setting OOZIE_HTTPS_KEYSTORE_FILE: /root/.keystore
Setting OOZIE_HTTPS_KEYSTORE_PASS: password
Setting CATALINA_OUT: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/logs/catalina.out
Setting CATALINA_PID: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/temp/oozie.pid
Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/logs/derby.log
Adding to CATALINA_OPTS: -Doozie.home.dir=/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2 -Doozie.config.dir=/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/conf -Doozie.log.dir=/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/logs -Doozie.data.dir=/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/data -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=dm4 -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443 -Doozie.base.url=http://dm4:11000/oozie -Doozie.https.keystore.file=/root/.keystore -Doozie.https.keystore.pass=password -Djava.library.path=
Using CATALINA_BASE: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server
Using CATALINA_HOME: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server
Using CATALINA_TMPDIR: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/temp
Using JRE_HOME: /usr/lib/jvm/jdk1.6.0_43/jre
Using CLASSPATH: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/bin/bootstrap.jar
Using CATALINA_PID: /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/temp/oozie.pid
Jun 2, 2013 3:06:46 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path:
Jun 2, 2013 3:06:46 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-11000
Jun 2, 2013 3:06:46 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 338 ms
Jun 2, 2013 3:06:46 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jun 2, 2013 3:06:46 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.36
Jun 2, 2013 3:06:46 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor oozie.xml
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/webapps/oozie/WEB-INF/lib/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/webapps/oozie/WEB-INF/lib/slf4j-simple-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
log4j:WARN No appenders could be found for logger (org.apache.jasper.compiler.JspRuntimeContext).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Jun 2, 2013 3:06:50 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ROOT
Jun 2, 2013 3:06:50 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-11000
Jun 2, 2013 3:06:50 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 3976 ms
界面:
和例子界面类似,没有内容。
案例:
将/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2中的oozie-examples.tar.gz解压,出现examples文件夹
修改
/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/examples/map-reduce/job.properties文件为如下形式:
nameNode=hdfs://211.80.198.148:9000
jobTracker=211.80.198.148:9001
queueName=default
examplesRoot=examples
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/map-reduce
outputDir=map-reduce
将文件夹上传到集群
root@dm4:/home/hadoop# bin/hadoop fs -copyFromLocal /home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/examples /user/root
运行命令
root@dm4:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2# bin/oozie job -oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -run
运行结果:
job: 0000000-130602150647008-oozie-root-W
不上传会出现如下错误:
root@dm4:/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2# bin/oozie job -oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -run
Error: E0504 : E0504: App directory [hdfs://211.80.198.148:9000/user/root/examples/apps/map-reduce] does not exist
结果
PS:中间出现的一些错误
ERROR: Oozie could not be started
REASON: java.lang.NoClassDefFoundError: org/apache/hadoop/util/ReflectionUtils
将hadoop的jar包(主目录下和lib目录下)拷贝到如下位置
/home/oozie/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/webapps/oozie/WEB-INF/lib
之后,运行正常。
参考:
1、http://angelosun.iteye.com/blog/1835881
2、http://practicalcloudcomputing.com/post/26337621577/installing-and-running-apache-oozie-3-2-x-and-possibly
3、http://wsysisibeibei.blog.163.com/blog/static/16536806120126233155505/
转载请保留:http://www.cnblogs.com/Deron/archive/2013/06/02/3111178.html