编译 hadoop-2.0.0-cdh4.2
下载 tar 包
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH-Version-and-Packaging-Information/cdhvd_topic_6_1.html
安装maven 3.0.5 并配置环境变量 M2_HOME M2 加入到path
安装ant 配置ANT_HOME 加入到path
安装protobuf 2.4.1 cdh4.2 依赖的protobuf-2.4.0 而最新的protobuf-2.5.0不兼容
我在我的mba 下 OSX10.9.1
./configure —prefix=/usr/local
make
编译 protobuf-2.4.1 报错
MT message.lo -MD -MP -MF .deps/message.Tpo -c google/protobuf/message.cc -fno-common -DPIC -o .libs/message.o
google/protobuf/message.cc:130:60: error: implicit instantiation of undefined template 'std::__1::basic_istream<char,
std::__1::char_traits<char> >'
vim src/google/protobuf/message.cc
在
#include <google/protobuf/stubs/stl_util-inl.h>
下增加
#include <istream>
保存 重新make 成功
然后make install
配置环境变量将 /usr/local/protobuf/bin加入path
到 HADOOP/src下运行
mvn -e package -Pdist,native,docs -DskipTests -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true
其中 -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true 用于解决 连接https 网站时得验证失败错误 peer not authenticated
报错 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/Users/wangfeng/Documents/workspace/hadoop-2.0.0-cdh4.2.0/src/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/Users/wangfeng/Documents/workspace/hadoop-2.0.0-cdh4.2.0/src/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory
mac OS 通过homebrew 安装cmake
brew install cmake
如果没有 homebrew 可以通过以下命令安装：
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
报错：
Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2
[exec] JAVA_AWT_INCLUDE_PATH)
此处尝试 export JAVA_INCLUDE_PATH =
依然通不过 索性 去掉了native和 doc
mvn -e package -Pdist -DskipTests -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true
编译成功 注意 从这里默认只能编译出hadoop mr2
cdh4 系列mr1 采用的0.20.3 需要的可以到
http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/4.2.0/SRPMS/ 下载源码rpm 获得
只有mr的代码