ZooKeeper配置引起Hbase和Kafka启动异常

时间:2023-01-26 08:23:50

使用ambari部署的HDP大数据平台,启动Hbase和Kafka的时候出现问题。
HBASE报错:
ZooKeeper配置引起Hbase和Kafka启动异常

2018-01-29 10:55:07,510 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2018-01-29 10:55:07,523 - File['/var/lib/ambari-agent/tmp/hbaseSmokeVerify.sh'] {'content': StaticFile('hbaseSmokeVerify.sh'), 'mode': 0755}
2018-01-29 10:55:07,531 - Writing File['/var/lib/ambari-agent/tmp/hbaseSmokeVerify.sh'] because it doesn't exist
2018-01-29 10:55:07,531 - Changing permission for /var/lib/ambari-agent/tmp/hbaseSmokeVerify.sh from 644 to 755
2018-01-29 10:55:07,537 - File['/var/lib/ambari-agent/tmp/hbase-smoke.sh'] {'content': Template('hbase-smoke.sh.j2'), 'mode': 0755}
2018-01-29 10:55:07,537 - Writing File['/var/lib/ambari-agent/tmp/hbase-smoke.sh'] because it doesn't exist
2018-01-29 10:55:07,537 - Changing permission for /var/lib/ambari-agent/tmp/hbase-smoke.sh from 644 to 755
2018-01-29 10:55:07,538 - Execute[' /usr/hdp/current/hbase-client/bin/hbase --config /usr/hdp/current/hbase-client/conf shell /var/lib/ambari-agent/tmp/hbase-smoke.sh && /var/lib/ambari-agent/tmp/hbaseSmokeVerify.sh /usr/hdp/current/hbase-client/conf id1bacba0a_date552918 /usr/hdp/current/hbase-client/bin/hbase'] {'logoutput': True, 'tries': 6, 'user': 'ambari-qa', 'try_sleep': 5}
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.2.0-2950/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-01-29 10:55:29,463 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
2018-01-29 10:55:29,465 WARN [main] zookeeper.ZKUtil: hconnection-0x79a7bfbc0x0, quorum=jgb01.domain:2181,jgb02.domain:2181,jgb03.domain:2181, baseZNode=/hbase-unsecure Unable to set watcher on znode (/hbase-unsecure/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase-unsecure/hbaseid
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:541)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:879)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:635)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362)
at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148)
at org.jruby.RubyClass.newInstance(RubyClass.java:822)
at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535)
at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at usr.hdp.$2_dot_3_dot_2_dot_0_minus_2950.hbase.bin.hirb.__file__(/usr/hdp/2.3.2.0-2950/hbase/bin/hirb.rb:131)
at usr.hdp.$2_dot_3_dot_2_dot_0_minus_2950.hbase.bin.hirb.load(/usr/hdp/2.3.2.0-2950/hbase/bin/hirb.rb)
at org.jruby.Ruby.runScript(Ruby.java:697)
at org.jruby.Ruby.runScript(Ruby.java:690)
at org.jruby.Ruby.runNormally(Ruby.java:597)
at org.jruby.Ruby.runFromMain(Ruby.java:446)
at org.jruby.Main.doRunFromMain(Main.java:369)
at org.jruby.Main.internalRun(Main.java:258)
at org.jruby.Main.run(Main.java:224)
at org.jruby.Main.run(Main.java:208)
at org.jruby.Main.main(Main.java:188)

kafka日志报错:
ZooKeeper配置引起Hbase和Kafka启动异常

[2018-01-29 12:00:20,418] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 25000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:143)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:121)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:88)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:223)
at kafka.server.KafkaServer.startup(KafkaServer.scala:112)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
at kafka.Kafka$.main(Kafka.scala:72)
at kafka.Kafka.main(Kafka.scala)
[2018-01-29 12:00:20,421] INFO shutting down (kafka.server.KafkaServer)
[2018-01-29 12:00:20,423] INFO shut down completed (kafka.server.KafkaServer)
[2018-01-29 12:00:20,424] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 25000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1205)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:143)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:121)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:88)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:223)
at kafka.server.KafkaServer.startup(KafkaServer.scala:112)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:29)
at kafka.Kafka$.main(Kafka.scala:72)
at kafka.Kafka.main(Kafka.scala)
[2018-01-29 12:00:20,425] INFO shutting down (kafka.server.KafkaServer)
[2018-01-29 12:59:22,462] INFO starting (kafka.server.KafkaServer)
[2018-01-29 12:59:22,464] INFO Connecting to zookeeper on jgb02.domain:2181,jgb03.domain:2181,jgb01.domain:2181 (kafka.server.KafkaServer)
[2018-01-29 12:59:22,878] INFO Loading logs. (kafka.log.LogManager)
[2018-01-29 12:59:22,885] INFO Logs loading complete. (kafka.log.LogManager)
[2018-01-29 12:59:22,885] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2018-01-29 12:59:22,888] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2018-01-29 12:59:22,922] INFO Awaiting socket connections on jgb03.domain:6667. (kafka.network.Acceptor)
[2018-01-29 12:59:22,924] INFO [Socket Server on Broker 1001], Started 1 acceptor threads (kafka.network.SocketServer)
[2018-01-29 12:59:22,945] INFO [ExpirationReaper-1001], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-01-29 12:59:22,946] INFO [ExpirationReaper-1001], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-01-29 12:59:23,007] INFO 1001 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2018-01-29 12:59:23,094] INFO [ExpirationReaper-1001], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-01-29 12:59:23,097] INFO [ExpirationReaper-1001], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-01-29 12:59:23,099] INFO [ExpirationReaper-1001], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-01-29 12:59:23,100] INFO New leader is 1001 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2018-01-29 12:59:23,112] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2018-01-29 12:59:23,131] INFO Registered broker 1001 at path /brokers/ids/1001 with addresses: PLAINTEXT -> EndPoint(jgb03.domain,6667,PLAINTEXT) (kafka.utils.ZkUtils$)
[2018-01-29 12:59:23,140] INFO [Kafka Server 1001], started (kafka.server.KafkaServer)

搜索了很久才找到一个有用的答案,记录下。
ZooKeeper配置超时:
修改ZooKeeper配置文件zoo.cfg ,增加一个超时配置。
timeout = 100000
重启服务恢复正常。