hbase集群的搭建(完全分布式)

时间:2024-10-06 07:03:20

说明

zookeeper和hadoop在头两篇文章已经介绍了,本文基于之前安装的集群安装hbase,安装的节点如下(标红的为本次安装):

机器

安装软件

进程

focuson1

zookeeper;hadoop namenode;hadoop DataNode;hbase master;hbase regionrerver

JournalNode; DataNode; QuorumPeerMain; NameNode; NodeManager;DFSZKFailoverController;HMaster;HRegionServer

focuson2

zookeeper;hadoop namenode;hadoop DataNode;yarn;hbase master;hbase regionrerver

NodeManager;ResourceManager;JournalNode; DataNode; QuorumPeerMain; NameNode; DFSZKFailoverController;HMaster;HRegionServer

focuson3

zookeeper;hadoop DataNode;yarn;hbase regionrerver

NodeManager;ResourceManager;JournalNode; DataNode; QuorumPeerMain;HRegionServer

注:zookeeper和hadoop戳链接:

zookeeper搭建

hadoop搭建

安装步骤

1、将包上传至用户家目录,解压

cd/usr/local/src/  
mkdir hbase
mv ~/hbase-1.4.3. .  
tar -xvf hbase-1.4.3.  
rm -f hbase-1.4.3.

2、配置类

配置文件一:$HBASE_HOME/conf/

export HBASE_MANAGES_ZK=false//默认hbase管理zookeeper,需要制定zookeeper的配置文件路径,修改成false,链接我们自己搭建的zookeeper集群
export JAVA_HOME=/usr/local/src/java/jdk1.7.0_51

配置文件二:$HBASE_HOME/conf/

<configuration>
<property>
    <name></name>
<!--链接两个ns1,ns1在hadoop集群中指向两个namenode,实现高可用,如果写定一个namenode,当该namenode为standby时,会报错
    该配置不需写端口-->
 <value>hdfs://ns1/hbase</value>
  </property>
<!--指定为分布式-->
 <property>
    <name></name>
    <value>true</value>
  </property>
<!--指定两个master,注册zookeeper监听,实现高可用-->
 <property>
    <name></name>
    <value>focuson1:60000,focuson2:60000</value>
  </property>
<!--zookeeper集群-->
 <property>
    <name></name>
    <value>focuson1:2181,focuson2:2181,focuson3:2181</value>
  </property>
<!--该配置必须有,要不访问不了web界面-->
 <property>
    <name></name>
    <value>60010</value>
  </property>
</configuration>

配置三:把hadoop的文件考到hbase的配置路径下

cp /usr/local/src/hadoop/hadoop-2.6.0/etc/hadoop/ /usr/local/src/hbase/hbase-1.4.3/conf/

3、启动

在focuson1上执行(启动focuson1的master,以及三台regionserver):

[root@focuson1 hbase-1.4.3]# ./bin/ 
running master, logging to /usr/local/src/hbase/hbase-1.4.3/logs/hbase-root-master-focuson1.out
focuson3: running regionserver, logging to /usr/local/src/hbase/hbase-1.4.3/bin/../logs/hbase-root-regionserver-focuson3.out
focuson1: running regionserver, logging to /usr/local/src/hbase/hbase-1.4.3/bin/../logs/hbase-root-regionserver-focuson1.out
focuson2: running regionserver, logging to /usr/local/src/hbase/hbase-1.4.3/bin/../logs/hbase-root-regionserver-focuson2.out
在focuson2上执行(只启动一个master,不会启动regionServer)
[root@focuson2 hbase-1.4.3]# ./bin/ 
running master, logging to /usr/local/src/hbase/hbase-1.4.3/bin/../logs/hbase-root-master-focuson2.out
focuson3: regionserver running as process 6705. Stop it first.
focuson1: regionserver running as process 17806. Stop it first.
focuson2: regionserver running as process 13949. Stop it first.

4 验证。

验证一:发现focuson1为active,focuson2为standby。

kill掉focuson1的hmaster进程,发现focuson2为active(不贴图了)

验证二:无论namenode为哪一个启动,hbase都能正常运行。

5、连接hdfs时出现的几个小问题。

问题一.不链接两个namenode实现高可用时,当链接focuson1,而此时focuson1正好为standby状态时,会报下面的错:

2018-05-02 00:01:20,746 FATAL [focuson1:16000.activeMasterManager] : Failed to become active master
(): Operation category READ is not supported in state standby
	at (:87)
	at $(:1719)
	at (:1350)
	at (:4132)
	at (:838)
	at (:821)
	at $ClientNamenodeProtocol$2.callBlockingMethod()
	at $Server$ProtoBufRpcInvoker.call(:619)
	at $Server.call(:962)

问题二、当配置高可用之后,Hmaster正常,HregionServer出现如下错误:

2018-05-02 01:36:41,083 ERROR [main] : Region server exiting
: Failed construction of Regionserver: class 
        at (:2812)
        at .start(:64)
        at .run(:87)
        at .run(:70)
        at (:127)
        at (:2827)
Caused by: 
        at 0(Native Method)
        at (:57)
        at (:45)
        at (:526)
        at (:2810)
        ... 5 more
Caused by: : : ns1
        at (:378)
        at (:320)
        at (:176)
        at .<init>(:678)
        at .<init>(:619)
        at .initialize(:149)
        at (:2669)
        at .access$200(:94)
        at $(:2703)
        at $Cache.get(:2685)
        at .get(:373)
        at (:295)
        at (:1037)
        at (:1080)
        at (:1062)
        at (:659)
        at .<init>(:602)
        ... 10 more
Caused by: : ns1
        ... 27 more

此时使用上述配置三完美解决!

---

完事!