转载于千与的 http://blog.csdn.net/shirdrn/article/details/7183503

ZooKeeper是一个分布式开源框架，提供了协调分布式应用的基本服务，它向外部应用暴露一组通用服务——分布式同步（Distributed Synchronization）、命名服务（Naming Service）、集群维护（Group Maintenance）等，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。ZooKeeper本身可以以Standalone模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。

有关ZooKeeper的介绍，网上很多，也可以参考文章后面，我整理的一些相关链接。

下面，我们简单说明一下ZooKeeper的配置。

ZooKeeper Standalone模式

从Apache网站上（zookeeper.apache.org）下载ZooKeeper软件包，我选择了3.3.4版本的（zookeeper-3.3.4.tar.gz），在一台Linux机器上安装非常容易，只需要解压缩后，简单配置一下即可以启动ZooKeeper服务器进程。

将zookeeper-3.3.4/conf目录下面的 zoo_sample.cfg修改为zoo.cfg，配置文件内容如下所示：

tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2

上面各个配置参数的含义也非常简单，引用如下所示：

tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.
clientPort —— the port to listen for client connections

下面启动ZooKeeper服务器进程：

cd zookeeper-3.3.4/
bin/zkServer.sh start

通过jps命令可以查看ZooKeeper服务器进程，名称为QuorumPeerMain。

在客户端连接ZooKeeper服务器，执行如下命令：

bin/zkCli.sh -server dynamic:2181

上面dynamic是我的主机名，如果在本机执行，则执行如下命令即可：

bin/zkCli.sh

客户端连接信息如下所示：

[plain] view plain copy

hadoop@master:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server dynamic:2181
Connecting to dynamic:2181
2012-01-08 21:30:06,178 - INFO [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 21:30:06,188 - INFO [main:Environment@97] - Client environment:host.name=master
2012-01-08 21:30:06,191 - INFO [main:Environment@97] - Client environment:java.version=1.6.0_30
2012-01-08 21:30:06,194 - INFO [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2012-01-08 21:30:06,200 - INFO [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 21:30:06,203 - INFO [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 21:30:06,206 - INFO [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 21:30:06,213 - INFO [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2012-01-08 21:30:06,216 - INFO [main:Environment@97] - Client environment:java.compiler=<NA>
2012-01-08 21:30:06,235 - INFO [main:Environment@97] - Client environment:os.name=Linux
2012-01-08 21:30:06,244 - INFO [main:Environment@97] - Client environment:os.arch=i386
2012-01-08 21:30:06,246 - INFO [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2012-01-08 21:30:06,251 - INFO [main:Environment@97] - Client environment:user.name=hadoop
2012-01-08 21:30:06,254 - INFO [main:Environment@97] - Client environment:user.home=/home/hadoop
2012-01-08 21:30:06,255 - INFO [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 21:30:06,264 - INFO [main:ZooKeeper@379] - Initiating client connection, connectString=dynamic:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2012-01-08 21:30:06,339 - INFO [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server dynamic/192.168.0.107:2181
Welcome to ZooKeeper!
2012-01-08 21:30:06,397 - INFO [main-SendThread(dynamic:2181):ClientCnxn$SendThread@950] - Socket connection established to dynamic/192.168.0.107:2181, initiating session
JLine support is enabled
2012-01-08 21:30:06,492 - INFO [main-SendThread(dynamic:2181):ClientCnxn$SendThread@739] - Session establishment complete on server dynamic/192.168.0.107:2181, sessionid = 0x134b9b714f9000c, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: dynamic:2181(CONNECTED) 0]

接着，可以使用help查看Zookeeper客户端可以使用的基本操作命令。

ZooKeeper Distributed模式

ZooKeeper分布式模式安装（ZooKeeper集群）也比较容易，这里说明一下基本要点。

首先要明确的是，ZooKeeper集群是一个独立的分布式协调服务集群，“独立”的含义就是说，如果想使用ZooKeeper实现分布式应用的协调与管理，简化协调与管理，任何分布式应用都可以使用，这就要归功于Zookeeper的数据模型（Data Model）和层次命名空间（Hierarchical Namespace）结构，详细可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在设计你的分布式应用协调服务时，首要的就是考虑如何组织层次命名空间。

下面说明分布式模式的安装配置，过程如下所示：

第一步：主机名称到IP地址映射配置

ZooKeeper集群中具有两个关键的角色：Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务，集群中每个结点之间都互相连接，所以，在配置的ZooKeeper集群的时候，每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。

例如，我的ZooKeeper集群中每个结点的配置，以slave-01为例，/etc/hosts内容如下所示：

192.168.0.179 slave-01
192.168.0.178 slave-02
192.168.0.177 slave-03

ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中，只有一个Leader，其他的都是Follower，如果ZooKeeper集群在运行过程中Leader出了问题，系统会采用该算法重新选出一个Leader。因此，各个结点之间要能够保证互相连接，必须配置上述映射。

ZooKeeper集群启动的时候，会首先选出一个Leader，在Leader election过程中，某一个满足选举算的结点就能成为Leader。整个集群的架构可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。

第二步：修改ZooKeeper配置文件

在其中一台机器（slave-01）上，解压缩zookeeper-3.3.4.tar.gz，修改配置文件conf/zoo.cfg，内容如下所示：

tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=slave-01:2888:3888
server.2=slave-02:2888:3888
server.3=slave-03:2888:3888

上述配置内容说明，可以参考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。

第三步：远程复制分发安装文件

上面已经在一台机器slave-01上配置完成ZooKeeper，现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下：

cd /home/hadoop/installation/
scp -r zookeeper-3.3.4/ hadoop@slave-02:/home/hadoop/installation/
scp -r zookeeper-3.3.4/ hadoop@slave-03:/home/hadoop/installation/

第四步：设置myid

在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字，例如：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-02:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-03:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid

按照上述进行配置即可。

第五步：启动ZooKeeper集群

在ZooKeeper集群的每个结点上，执行启动ZooKeeper服务的脚本，如下所示：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start

以结点slave-01为例，日志如下所示：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out
2012-01-08 06:51:19,117 - INFO [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
2012-01-08 06:51:19,133 - INFO [main:QuorumPeerConfig@310] - Defaulting to majority quorums
2012-01-08 06:51:19,167 - INFO [main:QuorumPeerMain@119] - Starting quorum peer
2012-01-08 06:51:19,227 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
2012-01-08 06:51:19,277 - INFO [main:QuorumPeer@819] - tickTime set to 2000
2012-01-08 06:51:19,278 - INFO [main:QuorumPeer@830] - minSessionTimeout set to -1
2012-01-08 06:51:19,279 - INFO [main:QuorumPeer@841] - maxSessionTimeout set to -1
2012-01-08 06:51:19,281 - INFO [main:QuorumPeer@856] - initLimit set to 5
2012-01-08 06:51:19,347 - INFO [Thread-1:QuorumCnxManager$Listener@473] - My election bind port: 3888
2012-01-08 06:51:19,393 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@621] - LOOKING
2012-01-08 06:51:19,396 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@663] - New election. My id = 1, Proposed zxid = 0
2012-01-08 06:51:19,400 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2012-01-08 06:51:19,416 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:19,420 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:19,612 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:19,615 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:19,616 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 400
2012-01-08 06:51:20,019 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,021 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,022 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 800
2012-01-08 06:51:20,825 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,827 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,828 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 1600
2012-01-08 06:51:22,435 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:22,439 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:22,441 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 3200
2012-01-08 06:51:22,945 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
2012-01-08 06:51:22,946 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@721] - Updating proposal
2012-01-08 06:51:22,949 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:22,951 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2012-01-08 06:51:23,156 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@643] - FOLLOWING
2012-01-08 06:51:23,170 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@80] - TCP NoDelay set to: true
2012-01-08 06:51:23,206 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 06:51:23,207 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:host.name=slave-01
2012-01-08 06:51:23,207 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.version=1.6.0_30
2012-01-08 06:51:23,208 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
2012-01-08 06:51:23,208 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 06:51:23,209 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.io.tmpdir=/tmp
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.compiler=<NA>
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.name=Linux
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.arch=i386
2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.version=3.0.0-14-generic
2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.name=hadoop
2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.home=/home/hadoop
2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 06:51:23,223 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@151] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
2012-01-08 06:51:23,339 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@294] - Getting a snapshot from leader
2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@325] - Setting leader epoch 1
2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@254] - Snapshotting: 0
2012-01-08 06:51:25,511 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
2012-01-08 06:51:42,584 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)

我启动的顺序是slave-01>slave-02>slave-03，由于ZooKeeper集群启动的时候，每个结点都试图去连接集群中的其它结点，先启动的肯定连不上后面还没启动的，所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到，集群在选出一个Leader后，最后稳定了。

其他结点可能也出现类似问题，属于正常。

第六步：安装验证

可以通过ZooKeeper的脚本来查看启动状态，包括集群中各个结点的角色（或是Leader，或是Follower），如下所示，是在ZooKeeper集群中的每个结点上查询的结果：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower
hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: leader
hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower

通过上面状态查询结果可见，slave-02是集群的Leader，其余的两个结点是Follower。

另外，可以通过客户端脚本，连接到ZooKeeper集群上。对于客户端来说，ZooKeeper是一个整体（ensemble），连接到ZooKeeper集群实际上感觉在独享整个集群的服务，所以，你可以在任何一个结点上建立到服务集群的连接，例如：

hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server slave-01:2181
Connecting to slave-01:2181
2012-01-08 07:14:21,068 - INFO [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 07:14:21,080 - INFO [main:Environment@97] - Client environment:host.name=slave-03
2012-01-08 07:14:21,085 - INFO [main:Environment@97] - Client environment:java.version=1.6.0_30
2012-01-08 07:14:21,089 - INFO [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2012-01-08 07:14:21,095 - INFO [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 07:14:21,104 - INFO [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 07:14:21,111 - INFO [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 07:14:21,116 - INFO [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2012-01-08 07:14:21,124 - INFO [main:Environment@97] - Client environment:java.compiler=<NA>
2012-01-08 07:14:21,169 - INFO [main:Environment@97] - Client environment:os.name=Linux
2012-01-08 07:14:21,175 - INFO [main:Environment@97] - Client environment:os.arch=i386
2012-01-08 07:14:21,177 - INFO [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2012-01-08 07:14:21,185 - INFO [main:Environment@97] - Client environment:user.name=hadoop
2012-01-08 07:14:21,188 - INFO [main:Environment@97] - Client environment:user.home=/home/hadoop
2012-01-08 07:14:21,190 - INFO [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 07:14:21,197 - INFO [main:ZooKeeper@379] - Initiating client connection, connectString=slave-01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2012-01-08 07:14:21,305 - INFO [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server slave-01/192.168.0.179:2181
Welcome to ZooKeeper!
2012-01-08 07:14:21,376 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@950] - Socket connection established to slave-01/192.168.0.179:2181, initiating session
JLine support is enabled
[zk: slave-01:2181(CONNECTING) 0] 2012-01-08 07:14:21,872 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@739] - Session establishment complete on server slave-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: slave-01:2181(CONNECTED) 0] ls /
[zookeeper]

当前根路径为/zookeeper。

总结说明

主机名与IP地址映射配置问题

启动ZooKeeper集群时，如果ZooKeeper集群中slave-01结点的日志出现如下错误：

java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:37:46,026 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 6400
2012-01-08 06:37:57,431 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/202.106.199.35:3888
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:38:02,442 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/202.106.199.35:3888

很显然，slave-01在启动时连接集群中其他结点（slave-02、slave-03）时，主机名映射的IP与我们实际配置的不一致，所以集群中各个结点之间无法建立链路，整个ZooKeeper集群启动是失败的。

上面错误日志中slave-02/202.106.199.35:3888实际应该是slave-02/202.192.168.0.178:3888就对了，但是在进行域名解析的时候映射有问题，修改每个结点的/etc/hosts文件，将ZooKeeper集群中所有结点主机名到IP地址的映射配置上。

附件：参考Zookeeper上的官方参考文档：

ZooKeeper

ZooKeeper：分布式应用程序的分布式协调服务

ZooKeeper：分布式应用程序的分布式协调服务

ZooKeeper是一种用于分布式应用程序的分布式，开源协调服务。它暴露了一组简单的原语，分布式应用程序可以基于实现更高级别的服务进行同步，配置维护以及组和命名。它被设计为易于编程，并使用在文件系统熟悉的目录树结构之后设计的数据模型。它运行在Java中，并且具有Java和C的绑定。

协调服务是非常难以正确的。他们特别容易出现种族条件和僵局等错误。ZooKeeper背后的动机是缓解分布式应用程序从头开始执行协调服务的责任。

设计目标

ZooKeeper很简单 ZooKeeper允许分布式进程通过与标准文件系统类似的共享分层命名空间相互协调。名称空间由ZooKeeper语法中的数据寄存器（称为znodes）组成，这些类似于文件和目录。与专为存储设计的典型文件系统不同，ZooKeeper数据保存在内存中，这意味着ZooKeeper可以实现高吞吐量和低延迟数量。

ZooKeeper实现对高性能，高可用性，严格有序的访问非常重要。ZooKeeper的性能方面意味着它可以在大型分布式系统中使用。可靠性方面使其不会成为单点故障。严格的排序意味着复杂的同步原语可以在客户端实现。

ZooKeeper被复制。像它所协调的分布式进程一样，ZooKeeper本身也是被复制到一组称为合奏的主机上。

ZooKeeper服务

构成ZooKeeper服务的服务器必须彼此了解。他们维护状态的内存映像以及持久存储中的事务日志和快照。只要大多数服务器可用，ZooKeeper服务将可用。

客户端连接到一个ZooKeeper服务器。客户端维护TCP连接，通过它发送请求，获取响应，获取观看事件并发送心跳。如果到服务器的TCP连接中断，客户端将连接到不同的服务器。

ZooKeeper订购。ZooKeeper将每个更新标记为反映所有ZooKeeper事务顺序的数字。后续操作可以使用该命令来实现更高级的抽象，例如同步原语。

ZooKeeper很快。它在“读主导”工作负载中特别快。ZooKeeper应用程序在数千台机器上运行，并且在写入次数比写入次数多10到1的情况下，性能最好。

数据模型和层次命名空间

ZooKeeper提供的名称空间与标准文件系统类似。名称是以斜杠（/）分隔的路径元素序列。ZooKeeper的名称空间中的每个节点都由路径标识。

ZooKeeper的分层命名空间

节点和短暂节点

与标准文件系统不同，ZooKeeper命名空间中的每个节点都可以与其相关联的数据以及子节点。就像有一个允许文件也是目录的文件系统。（ZooKeeper设计用于存储协调数据：状态信息，配置，位置信息等，因此存储在每个节点上的数据通常很小，字节到千字节范围）。我们使用术语 znode来表明我们正在谈论ZooKeeper数据节点。

Znodes维护统计结构，其中包括数据更改，ACL更改和时间戳的版本号，以允许缓存验证和协调更新。每次znode的数据发生变化时，版本号都会增加。例如，每当客户端检索数据时，它也会收到数据的版本。

存储在命名空间中的每个znode处的数据以原子方式读取和写入。读取获取与znode相关联的所有数据字节，写入替换所有数据。每个节点都有一个访问控制列表（ACL），它限制谁能做什么。

ZooKeeper还具有短暂节点的概念。只要创建znode的会话处于活动状态，就会存在这些znodes。当会话结束时，znode被删除。当您想实现[ tbd ]时，临时节点很有用。

有条件的更新和手表

ZooKeeper支持手表的概念。客户可以在znode上设置手表。当znode更改时，手表将被触发并移除。当手表触发时，客户端收到一个数据包，说明znode已经改变了。如果客户端与Zoo Keeper服务器之间的连接断开，客户端将收到本地通知。这些可以用于[tbd]。

保证

ZooKeeper非常快速，非常简单。由于其目标是为建设更为复杂的服务，如同步化的基础，它提供了一套保证。这些是：

顺序一致性 - 客户端的更新将按照发送的顺序进行应用。
原子性 - 更新成功或失败。没有部分结果。
单一系统映像 - 客户端将看到与服务器连接的服务器相同的服务视图。

可靠性 - 一旦应用了更新，它将从当前持续到客户端覆盖更新。

及时性 - 系统的客户端视图在一定时间内保证是最新的。

有关这些的更多信息以及如何使用它们，请参阅 [tbd]

简单的API

ZooKeeper的设计目标之一是提供一个非常简单的编程接口。因此，它仅支持以下操作：

创建: 在树中的某个位置创建一个节点
删除: 删除一个节点
存在: 测试节点是否存在于某个位置
获取数据: 从节点读取数据
设置数据: 将数据写入节点
得到孩子: 检索节点的子节点列表
同步: 等待数据传播

有关这些更深入的讨论，以及如何使用它们来实现更高层次的操作，请参考[ tbd ]

实施

ZooKeeper组件显示ZooKeeper服务的高级组件。除了请求处理器之外，组成ZooKeeper服务的每个服务器都会复制其每个组件的自己的副本。

ZooKeeper组件

复制数据库是包含整个数据树的内存数据库。将更新记录到磁盘以获取可恢复性，并将写入序列化到磁盘，然后才能应用到内存数据库。

每个ZooKeeper服务器都为客户端服务。客户端连接到一个服务器以提交irequests。从每个服务器数据库的本地副本服务器读取请求。更改服务状态，写入请求的请求由协议协议进行处理。

作为协议协议的一部分，客户端的所有写入请求都将转发到单个服务器，称为主管。称为关注者的其他ZooKeeper服务器从领导者接收消息提议，并同意消息传递。消息传递层负责替代领导者的失败，并与领导者同步追随者。

ZooKeeper使用自定义的原子消息协议。由于消息层是原子的，所以ZooKeeper可以保证本地副本不会发散。当领导者收到写请求时，它会计算要应用写入时系统的状态，并将其转换为捕获此新状态的事务。

用途

ZooKeeper的编程界面故意简单。然而，您可以实现更高阶的操作，例如同步原语，组成员资格，所有权等。一些分布式应用程序已使用它：[ tbd ：从白皮书和视频演示中添加用途。）有关详细信息，请参阅 [tbd]

性能

ZooKeeper的设计是高性能的。但是呢 Yahoo! Research的ZooKeeper开发团队的结果表明它是。（请参见ZooKeeper吞吐量作为读写比率的变化。）读取超出写入的应用程序中性能特别高，因为写入涉及同步所有服务器的状态。（协调服务通常情况下读取数量超过写入次数。）

ZooKeeper吞吐量随读写速率的变化而变化

图形ZooKeeper吞吐量作为读写比率变化是ZooKeeper 3.2版本在具有双2Ghz至强和两个SATA 15K RPM驱动器的服务器上运行的吞吐量图。一个驱动器用作专用的ZooKeeper日志设备。快照已写入OS驱动器。写入请求是1K写入，读取是1K读取。“服务器”表示ZooKeeper系统的大小，构成服务的服务器数量。大约使用其他30台服务器来模拟客户端。ZooKeeper系列被配置为使领导者不允许与客户端的连接。

注意

在3.2版本中，r / w性能比以前的3.1版本提高了约2倍。

基准也表明它也是可靠的。出现错误的可靠性显示部署如何响应各种故障。图中标有的事件如下：

追随者的失败和恢复
失败和恢复不同的追随者
领导失败
失败和恢复的两个追随者
另一位领导失败

可靠性

为了显示系统随着时间的推移，注意到我们运行了一个由7台机器组成的ZooKeeper服务。我们执行了与以前相同的饱和基准，但这次我们将写入百分比保持在30％，这是我们预期工作负荷的保守比例。

出现错误的可靠性

这是这个图表的一些重要观察。首先，如果追随者失败并迅速恢复，那么ZooKeeper尽管发生故障仍然能够维持高吞吐量。但也许更重要的是，领导选举算法可以使系统恢复得足够快，以防止吞吐量大幅下降。在我们的观察中，ZooKeeper需要不到200ms才能选出新的领导者。第三，随着追随者的恢复，ZooKeeper能够在开始处理请求后再次提高吞吐量。

ZooKeeper项目

ZooKeeper已经成功应用于许多工业应用。它在Yahoo!中用作Yahoo! Message Broker的协调和故障恢复服务，Yahoo! Message Broker是一个高度可扩展的发布订阅系统，用于管理数千个用于复制和数据传送的主题。它由用于Yahoo! crawler的获取服务使用，它还管理故障恢复。一些Yahoo!广告系统也使用ZooKeeper来实现可靠的服务。

鼓励所有用户和开发人员加入社区并贡献自己的专长。有关更多信息，请参阅Apache上的 Zookeeper项目。

参考链接

下面是我整理搜集的有关ZooKeeper相关内容的网址，可以学习参考。

中文链接：

英文链接：

秒客网

（转）ZooKeeper-3.3.4集群安装配置