关于 hadoop-2.2 配置中的 yarn.nodemanager.aux-services 项

时间:2021-06-01 18:20:01

在hadoop-2.0.3-alpha 的配置中,yarn.nodemanager.aux-services项的默认值是“mapreduce.shuffle”,但如果在hadoop-2.2 中继续使用这个值,NodeManager 会启动失败,在logs/yarn-biadmin-nodemanager-hostname.log里会报如下的错:

2014-01-19 22:35:04,309 FATAL org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Failed to initialize mapreduce.shuffle
java.lang.IllegalArgumentException: The ServiceName: mapreduce.shuffle set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:98)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:218)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:188)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:338)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:386)


解决方案

将yarn.nodemanager.aux-services项的值改为“mapreduce_shuffle”。