二：HDFS 命令指南

时间：2022-09-30 20:24:23
命令具体选项请参考：
 

1.HDFS命令通过bin/hdfs执行，语法如下：

 hdfs [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]
        配置文件          用户命令   基本选项           命令选项
COMMAND_OPTION Description
--config confdir Overwrites the default Configuration directory. Default is $HADOOP_HOME/conf.
GENERIC_OPTIONS The common set of options supported by multiple commands. Full list is here.
COMMAND_OPTIONS Various commands with their options are described in the following sections. The commands have been grouped into and .
 
2.GENREIC_OPTIONS基本命令如下：
GENERIC_OPTION Description
-conf <configuration file> Specify an application configuration file.  指定配置文件
-D <property>=<value> Use value for given property.                   指定属性
-jt <local> or <resourcemanager:port> Specify a ResourceManager. Applies only to job.   指定资源管理器
-files <comma separated list of files> Specify comma separated files to be copied to the map reduce cluster. Applies only to job.
-libjars <comma seperated list of jars> Specify comma separated jar files to include in the classpath. Applies only to job.
-archives <comma separated list of archives> Specify comma separated archives to be unarchived on the compute machines. Applies only to job.
 
3.用户命令
文件系统命令：hdfs dfs [GENERIC_OPTIONS] [COMMAND_OPTIONS] 参考： File System Shell Guide.
口令命令   ：hdfs fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>
文件检查命令：hdfs fsck [GENERIC_OPTIONS] <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-
            locks   -locations | -racks]]]] [-includeSnapshots]
版本信息   :hdfs version
 
4.管理命令
dfsadmi
平衡        ：hdfs balancer [-threshold <threshold>] [-policy <policy>]
启动回滚撤销datanode ： hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
dfsadmin管理：hdfs dfsadmin [GENERIC_OPTIONS]
mover数据迁移工具   :hdfs mover [-p <files/dirs> | -f <local file name>] 类似于balancer，会检查块是否满足存储要求
namenode管理：
Usage: hdfs namenode [-backup] |

          [-checkpoint] |

          [-format [-clusterid cid ] [-force] [-nonInteractive] ] |

          [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |

          [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |

          [-rollback] |

          [-rollingUpgrade <downgrade|rollback> ] |

          [-finalize] |

          [-importCheckpoint] |

          [-initializeSharedEdits] |

          [-bootstrapStandby] |

          [-recover [-force] ] |

          [-metadataVersion ]
 
secondary namenode:

hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]

COMMAND_OPTION Description
-checkpoint [force] Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If force is used, checkpoint irrespective of EditLog size.
-format Format the local storage during startup.
-geteditsize Prints the number of uncheckpointed transactions on the NameNode.
 
 
注意：
sbin/start-dfs.sh实际上就是启动了namenode datanode secondarynamenode zooker等
  --script "$bin/hdfs" start namenode $nameStartOpt
  --script "$bin/hdfs" start datanode $dataStartOpt
  --script "$bin/hdfs" start secondarynamenode
  --script "$bin/hdfs" start zkfc
通过SSH连接到nanenode上启动
5.存储策略
hdfs storagepolicies 列出策略
hdfs dfsadmin -setStoragePolicy <path> <policyName> 设置策略
hdfs dfsadmin -getStoragePolicy <path> 获取策略
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

<wiz_tmp_tag id="wiz-table-range-border" contenteditable="false" style="display: none;">

 
 
 
 

相关文章