在windows上构建hadoop 2.2。

I am trying to install hadoop on windows, it is so troublesome so far.

我正在尝试在windows上安装hadoop，到目前为止它太麻烦了。

here is error i keep getting

这是我一直得到的错误。

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (compile-ms-winutils) on project hadoop-common: Command execution failed. Cannot run program "msbuild" (in directory "C:\hdfs\hadoop-common-project\hadoop-common"): CreateProcess error=2, The system cannot find the file specified -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-common

I have msbuild.exe, windows sdk7.1, cygwin64, maven3.0.5, jdk7 all set up in environment on my windows 7 x64 box.

我有msbuild。exe、windows sdk7.1、cygwin64、maven3.0.5、jdk7都在我的windows 7 x64 box环境中设置。

This is hadoop 2.2.0 src which i downloaded from apache website.

这是hadoop 2.2.0 src，我是从apache网站下载的。

under system environment, i have set

在系统环境下，我设置了。

C:\Windows\Microsoft.NET\Framework64\v4.0.30319

this is the path to msbuild, but still doesn't work, does anyone know how to fix this?

这是通往msbuild的路径，但仍然不能工作，有人知道如何修复它吗?

thanks

谢谢

update: I changed pom.xml at hadoop-common-project\hadoop-common a little from

更新:我改变了pom。在hadoop- commonproject \hadoop-common中有一些xml。

<executable>msbuild</executable>

来

<executable>msbuild.exe</executable>

Now, I'm getting following error

现在，我得到了一个错误。

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (compile-ms-winutils) on project hadoop-common: Command execution failed. Process exited with an error: 1(Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-common

7 个解决方案

#1

I found the solution to this problem. I have VS 2012 on Windows 8.1 and faced this exact issue. I checked out the pom.xml under hadoop-common for that execution id and ran the following command

我找到了解决这个问题的办法。我在Windows 8.1上有VS 2012，面对这个问题。我检查了一下pom。用于执行id的hadoop-common下的xml并运行以下命令。

F:\HDP>msbuild F:\HDP\hadoop-common-project\hadoop-common\src\main\winutils\winutils.sln /nologo/p:Configuration=Release;OutDir=bin

It gave me the following exception

它给了我以下的例外。

C:\Program Files(x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.Cpp.Platform.targets(42,5): error MSB8020: The builds tools for Visual Studio 2010 (Platform Toolset ='v100') cannot be found.
To build using the v100 build tools, either click the Project menu or right-click the solution, and then select "Update VC++ Projects...". Install Visual Studio 2010 to build using the Visual Studio 2010 build tools.
[F:\HDP\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]

Since I do not have Visual Studio 2010 on my machine, the problem had to be in the solution. I opened the solution in VS 2012 and compiled it once. Executing maven again gave me another error but this time pointing to hadoop-common\src\main\native\native.sln. I opened that on VS 2012 too and no error.

由于我的机器上没有Visual Studio 2010，所以问题必须在解决方案中。我打开了VS 2012的解决方案，并编译了一次。执行maven再次给了我另一个错误，但这次指向的是maven -common\src\main\native\native.sln。我在VS 2012上也打开了它，没有错误。

Hope this helps

希望这有助于

Thanks Rohit

由于罗希特

#2

I have followed following steps to install Hadoop 2.2.0

我已经按照以下步骤安装了Hadoop 2.2.0。

Steps to build Hadoop bin distribution for Windows

为Windows构建Hadoop bin分布的步骤。

Download and install Microsoft Windows SDK v7.1.

下载并安装Microsoft Windows SDK v7.1。
Download and install Unix command-line tool Cygwin.

下载并安装Unix命令行工具Cygwin。
Download and install Maven 3.1.1.

下载并安装Maven 3.1.1。
Download Protocol Buffers 2.5.0 and extract to a folder (say c:\protobuf).

下载协议缓冲区2.5.0并提取到一个文件夹(比如c:\protobuf)。
Add Environment Variables JAVA_HOME, M2_HOME and Platform if not added already. Note : Variable name Platform is case sensitive. And value will be either x64 or Win32 for building on a 64-bit or 32-bit system. Edit Path Variable to add bin directory of Cygwin (say C:\cygwin64\bin), bin directory of Maven (say C:\maven\bin) and installation path of Protocol Buffers (say c:\protobuf).

添加环境变量JAVA_HOME, M2_HOME和平台，如果没有添加的话。注:变量名平台区分大小写。而且，在64位或32位系统上构建的值将是x64或Win32。编辑Path变量以添加Cygwin的bin目录(比如C:\cygwin64\bin)、Maven的bin目录(比如C:\ Maven \bin)和协议缓冲区的安装路径(比如C:\ protobuf)。
Download hadoop-2.2.0-src.tar.gz and extract to a folder having short path (say c:\hdfs) to avoid runtime problem due to maximum path length limitation in Windows.

下载hadoop-2.2.0-src.tar。gz和extract到一个有短路径的文件夹(比如c:\hdfs)，以避免由于Windows的最大路径长度限制而导致运行时问题。
Select Start --> All Programs --> Microsoft Windows SDK v7.1 and open Windows SDK 7.1 Command Prompt. Change directory to Hadoop source code folder (c:\hdfs). Execute mvn package with options -Pdist,native-win -DskipTests -Dtar to create Windows binary tar distribution.

选择Start——>所有程序——>微软Windows SDK v7.1和open Windows SDK 7.1命令提示。将目录更改为Hadoop源代码文件夹(c:\hdfs)。使用选项-Pdist,native-win - dskiptest -Dtar来执行mvn包，以创建Windows二进制tar分发包。
If everything goes well in the previous step, then native distribution hadoop-2.2.0.tar.gz will be created inside C:\hdfs\hadoop-dist\target\hadoop-2.2.0 directory.

如果在前面的步骤中一切顺利，那么本地分发hadoop-2.2.0.tar。gz将在C:\hdfs\hadoop-dist\target\hadoop-2.2.0目录中创建。

Install Hadoop

安装Hadoop

Extract hadoop-2.2.0.tar.gz to a folder (say c:\hadoop).

提取hadoop-2.2.0.tar。gz到一个文件夹(比如c:\hadoop)。
Add Environment Variable HADOOP_HOME and edit Path Variable to add bin directory of HADOOP_HOME (say C:\hadoop\bin).

添加环境变量hadoop home和edit Path变量来添加hadoop home的bin目录(比如C:\hadoop\bin)。

Configure Hadoop

配置Hadoop

C:\hadoop\etc\hadoop\core-site.xml

C:\ hadoop \ etc \ hadoop \ core-site.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://localhost:9000</value>
        </property>
</configuration>

C:\hadoop\etc\hadoop\hdfs-site.xml

C:\ hadoop \ etc \ hadoop \ hdfs-site.xml

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>file:/hadoop/data/dfs/namenode</value>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>file:/hadoop/data/dfs/datanode</value>
        </property>
</configuration>

C:\hadoop\etc\hadoop\mapred-site.xml

C:\ hadoop \ etc \ hadoop \ mapred-site.xml

<configuration>
        <property>
           <name>mapreduce.framework.name</name>
           <value>yarn</value>
        </property>
</configuration>

C:\hadoop\etc\hadoop\ yarn-site.xml

C:\ hadoop \ etc \ hadoop \ yarn-site.xml

<configuration>
        <property>
           <name>yarn.nodemanager.aux-services</name>
           <value>mapreduce_shuffle</value>
        </property>
        <property>
           <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
           <value>org.apache.hadoop.mapred.ShuffleHandler</value>
        </property>
</configuration>

Format namenode

namenode格式

For the first time only, namenode needs to be formatted.

第一次只需要格式化namenode。

C:\Users\abhijitg>cd c:\hadoop\bin 
c:\hadoop\bin>hdfs namenode –format

Start HDFS (Namenode and Datanode)

启动HDFS (Namenode和Datanode)

C:\Users\abhijitg>cd c:\hadoop\sbin
c:\hadoop\sbin>start-dfs

Start MapReduce aka YARN (Resource Manager and Node Manager)

启动MapReduce aka线程(资源管理器和节点管理器)

C:\Users\abhijitg>cd c:\hadoop\sbin
c:\hadoop\sbin>start-yarn
starting yarn daemons

Total four separate Command Prompt windows will be opened automatically to run Namenode, Datanode, Resource Manager, Node Manager

四个单独的命令提示符窗口将自动打开以运行Namenode、Datanode、资源管理器、节点管理器。

Reference : Build, Install, Configure and Run Apache Hadoop 2.2.0 in Microsoft Windows OS

参考:在Microsoft Windows操作系统中构建、安装、配置和运行Apache Hadoop 2.2.0。

#3

I had the same problem. I solved it by

我遇到了同样的问题。我解决了问题

opening a dos command prompt

打开dos命令提示符。
starting "c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat" which modifies the PATH variable and sets some other variables

启动“c:\程序文件(x86)\微软Visual Studio 10.0\VC\vcvarsall。这将修改路径变量并设置其他变量。
After that I started cygwin directly from this command prompt

C:\cygwin64\bin\mintty.exe -i /Cygwin-Terminal.ico -
在那之后，我从这个命令提示中直接启动cygwin C:\cygwin64\bin\mintty。exe - i / Cygwin-Terminal。ico -

Now the PATH is properly formatted and msbuild is found ...

现在，路径被正确格式化，msbuild被发现…

Hope this helps!

希望这可以帮助!

BTW: If it is complaining about the wrong version of msbuild maybe you have to install Visual Studio Express 2010. Or as I read elsewhere you can use VisualStudioExpress2012 and set VS100COMNTOOLS pointing to VS110COMNTOOLS.

顺便说一句，如果你在抱怨msbuild的错误版本，也许你需要安装Visual Studio Express 2010。或者，当我在其他地方阅读时，你可以使用VisualStudioExpress2012，并设置VS100COMNTOOLS指向VS110COMNTOOLS。

#4

You may need CMake.exe - get it using official link http://www.cmake.org/download/

你可能需要CMake。exe -使用官方链接http://www.cmake.org/download/。

Open file BUILDING.txt which located in unzipped hadoop-2.6.0-src.tar.gz or your version.
It say you how to build for Windows - check all of this:

打开的文件。txt位于解压缩的hadoop-2.6.0-src.tar中。广州或你的版本。它告诉你如何构建Windows——检查所有这些:

My BUILDING.txt says:

我的建筑。txt说:

Building on Windows
Requirements:

* Windows System
* JDK 1.6+
* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer
* Windows SDK or Visual Studio 2010 Professional
* Unix command-line tools from GnuWin32 or Cygwin: sh, mkdir, rm, cp, tar, gzip
* zlib headers (if building native code bindings for zlib)
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

If using Visual Studio, it must be Visual Studio 2010 Professional (not 2012).
Do not use Visual Studio Express.  It does not support compiling for 64-bit,
which is problematic if running a 64-bit system.  The Windows SDK is free to
download here:

http://www.microsoft.com/en-us/download/details.aspx?id=8279

#5

Even after following all the steps mentioned in the document and here, I got the following error:

即使遵循了文档中提到的所有步骤，我还是得到了以下错误:

Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec 
(compile-ms-winutils) on project hadoop-common

The detailed error said:

详细的错误说:

You are attempting to build a Win32 application from an x64 environment. 
If using the Windows 7.1 SDK build environment, type setenv /x86

So I added the following environment variable and that resolved the issue

因此，我添加了以下环境变量，并解决了这个问题。

Platform: x64

#6

It seems that there is problem with path..

4 .似乎道路有问题。

add the msbuild path to Environment Variable Path

将msbuild路径添加到环境变量路径。

#7

I was able to fix this issue by below two steps.

我可以通过以下两个步骤来解决这个问题。

1) Add the MsBuild path(C:\Windows\Microsoft.NET\Framework64\v4.0.30319) to Environment Variable Path

1)将MsBuild路径(C:\Windows\Microsoft.NET\Framework64\v4.0.30319)添加到环境变量路径中。

2) Installing Microsoft-Visual-Studio-2010-Professional http://download.cnet.com/Microsoft-Visual-Studio-2010-Professional/3000-2212_4-10618634.html

2)安装微软-视觉工作室- 2010 -专业http://download.cnet.com/microsoft -视觉工作室- 2010 professional/3000 - 2212 - _4 - 10618634. - html

Note : Before installing Visual-Studio-2010-Professional I tried Windows-SDK-7.1 which did not help me. So I uninstalled Windows-SDK-7.1 before installing Visual-Studio-2010-Professional. Also the link I provided for Visual-Studio-2010-Professional is a trial version.

注意:在安装visualstudio -2010- professional之前，我尝试了Windows-SDK-7.1，这对我没有帮助。所以在安装visualstudio -2010- professional之前，我卸载了Windows-SDK-7.1。我为visualstudio -2010- professional提供的链接也是一个试用版。

#1