win10编译Hadoop3.0.2源码遇到的坑

时间:2024-03-14 15:01:21

目录

背景

1、Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

3、Cannot run program "msbuild" (in directory "D:\hadoop-3.0.2-src\hadoop-common-project\hadoop-common"): CreateProcess error=2, 系统找不到指定的文件。

4、 Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part ...... @ 11:135 in D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml

5、 Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

编译结果

资源下载链接


背景

周五尝试用hive建表时,出现了问题,需要修改hadoop源码,而后编译。

但是在编译时,遇到了不少问题,周日耗时一天,排查完毕,编译通过

所有资源网盘链接见最后,请按需使用

1、Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version

 

解决方法:安装protobuf2.5-win

1、下载zip压缩包

2、解压压缩包,然后把解压路径添加到环境变量path里

 

2、 Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command

 

javah出现了问题,解决方法:

修改源码目录下hadoop-common-project\hadoop-common\pom.xml文件,把所有的javahPath标签内容改成javah的绝对路径

 

3、Cannot run program "msbuild" (in directory "D:\hadoop-3.0.2-src\hadoop-common-project\hadoop-common"): CreateProcess error=2, 系统找不到指定的文件。

     

解决方法:安装VS Code,然后把msbuild可执行文件的绝对路径添加到path里(C:\Program Files (x86)\MSBuild\14.0\Bin)

 

4、 Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1

[ERROR] around Ant part ...<exec failοnerrοr="true" dir="D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="msbuild">... @ 11:135 in D:\hadoop-3.0.2-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml

 

     cmake和MsBuild(MSB1009找不到)莫名报错,暂时在hadoop-hdfs-project\hadoop-hdfs-native-client\pom.xml中,屏蔽它们的错误

     win10编译Hadoop3.0.2源码遇到的坑

 

5、 Could not resolve dependencies for project org.apache.hadoop:hadoop-resourceestimator:jar:3.0.2: Could not transfer artifact org.ojalgo:ojalgo:jar:43.0 from/to nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public): GET request of: org/ojalgo/ojalgo/43.0/ojalgo-43.0.jar from nexus-aliyun failed: Connection reset

 

     解决方法:把ojalgo-43.0.jar下载下来,复制到本地maven仓库文件夹的org\ojalgo\ojalgo\jar\43.0\目录下

 

编译结果

win10编译Hadoop3.0.2源码遇到的坑

资源下载链接

链接:https://pan.baidu.com/s/10PY2fhlBqJ0XNQiwlMpQIA 
提取码:l4bb 
 

包括hadoop3.0.2源码压缩包,apache-maven-3.6.1-bin.zip,cmake-3.14.5-win32-x86.msi,ojalgo-43.0.jar,protoc-2.5.0-win32.zip,VSCodeSetup-x64-1.25.1.exe,zlib-1.2.3.exe