一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

时间:2023-03-08 18:03:39
一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

Storm的官方网址:http://storm.apache.org/index.html

:集群部署的基本流程(基本套路):
集群部署的流程:下载安装包、解压安装包、修改配置文件、分发安装包、启动集群;

1:安装一个zookeeper集群,之前已经部署过,这里省略,贴一下步骤;

安装配置zooekeeper集群:
        1.1:解压
            tar -zxvf zookeeper-3.4.5.tar.gz
        1.2:修改配置
            cd /home/hadoop/zookeeper-3.4.5/conf/
            cp zoo_sample.cfg zoo.cfg
            vim zoo.cfg
            修改:dataDir=/home/hadoop/zookeeper-3.4.5/tmp
            在最后添加:
            server.1=master:2888:3888
            server.2=slaver1:2888:3888
            server.3=slaver2:2888:3888
            保存退出
            然后创建一个tmp文件夹
            mkdir /home/hadoop/zookeeper-3.4.5/tmp
            再创建一个空文件
            touch /home/hadoop/zookeeper-3.4.5/tmp/myid
            最后向该文件写入ID
            echo 1 > /home/hadoop/zookeeper-3.4.5/tmp/myid
        1.3将配置好的zookeeper拷贝到其他节点:
            scp -r /home/hadoop/zookeeper-3.4.5/ slaver1:/home/hadoop/
            scp -r /home/hadoop/zookeeper-3.4.5/ slaver2:/home/hadoop/
            
            注意:修改slaver1、slaver2对应/home/hadoop/zookeeper-3.4.5/tmp/myid内容
            slaver1:
                echo 2 > /home/hadoop/zookeeper-3.4.5/tmp/myid
            slaver2:
                echo 3 > /home/hadoop/zookeeper-3.4.5/tmp/myid

2、上传storm的安装包,解压缩:

[root@master hadoop]# tar -zxvf apache-storm-0.9.2-incubating.tar.gz

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

3、修改配置文件storm.yaml:

可以创建一个软连接,方便操作storm:[root@master soft]# ln -s apache-storm-0.9.2-incubating storm

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

修改内容如下所示,下面两个配置均可:

#指定storm使用的zk集群
storm.zookeeper.servers:
- "zk01"
- "zk02"
- "zk03"
#指定storm集群中的nimbus节点所在的服务器
nimbus.host: "storm01"
#指定nimbus启动JVM最大可用内存大小
nimbus.childopts: "-Xmx1024m"
#指定supervisor启动JVM最大可用内存大小
supervisor.childopts: "-Xmx1024m"
#指定supervisor节点上,每个worker启动JVM最大可用内存大小
worker.childopts: "-Xmx768m"
#指定ui启动JVM最大可用内存大小,ui服务一般与nimbus同在一个节点上。
ui.childopts: "-Xmx768m"
#指定supervisor节点上,启动worker时对应的端口号,每个端口对应槽,每个槽位对应一个worker
supervisor.slots.ports:
-
-
-
-

此次配置使用下面的,进行集群安装:

#所使用的zookeeper集群主机
storm.zookeeper.servers:
- "master"
- "slaver1"
- "slaver2" #nimbus所在的主机名
nimbus.host: "master" supervisor.slots.ports
-6701
-6702
-6703
-6704
-6705

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

然后将master修改好的storm发送到slaver1,slaver2:

[root@master hadoop]# scp -r apache-storm-0.9.2-incubating/ slaver1:/home/hadoop/

[root@master hadoop]# scp -r apache-storm-0.9.2-incubating/ slaver2:/home/hadoop/

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

4:启动storm集群,首先启动你的Zookeeper集群,然后再启动你的storm集群哈。

启动Zookeeper集群:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

然后启动Storm集群:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

启动storm
在nimbus主机上,在nimbus.host所属的机器上启动 nimbus服务:
  nohup ./storm nimbus >/dev/null >& & 或者使用命令:nohup ./storm nimbus &
  或者使用命令:./storm nimbus > ../logs/info 2>&1 &  (注释:将启动信息,错误信息,标准信息都打印到如此目录里面)
在nimbus.host所属的机器上启动ui服务:
  nohup ./storm ui >/dev/null >& & 或者使用命令:nohup ./storm ui & 在supervisor主机上,在其它个点击上启动supervisor服务:
  nohup ./storm supervisor >/dev/null >& & 或者使用命令:nohup ./storm supervisor &
  或者使用命令:./storm supervisor > ../logs/info 2>&1 (注释:将启动信息,错误信息,标准信息都打印到如此目录里面)

注意,解释:

1>/dev/null:代表标准输入到这个目录;

2>&1:代表标准输出也到这个目录下面;

&:代表这个是后台运行;

如下启动storm方便观察,最后一行是卡住不动的哦:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

查看进程如下所示:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

可以启动一下storm的ui查看:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

查看一下进程如:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

启动ui以后可以在浏览器访问,如:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

启动supervisor

[root@slaver1 bin]# ./storm supervisor

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

然后可以启动剩下的storm:

[root@slaver2 bin]# ./storm supervisor

启动以后可以查看进程jps的启动情况,然后可以去浏览器查看自己http://192.168.3.129:8080/index.html的页面各个启动情况,如supervisor等等。

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

5:Storm常用操作命令:

:有许多简单且有用的命令可以用来管理拓扑,它们可以提交、杀死、禁用、再平衡拓扑。
提交任务命令格式:storm jar 【jar路径】 【拓扑包名.拓扑类名】 【拓扑名称】
:bin/storm jar examples/storm-starter/storm-starter-topologies-0.9..jar storm.starter.WordCountTopology wordcount
杀死任务命令格式:storm kill 【拓扑名称】 -w (执行kill命令时可以通过-w [等待秒数]指定拓扑停用以后的等待时间)
:storm kill topology-name -w
停用任务命令格式:storm deactivte 【拓扑名称】
:storm deactivte topology-name
:我们能够挂起或停用运行中的拓扑。当停用拓扑时,所有已分发的元组都会得到处理,但是spouts的nextTuple方法不会被调用。销毁一个拓扑,可以使用kill命令。它会以一种安全的方式销毁一个拓扑,首先停用拓扑,在等待拓扑消息的时间段内允许拓扑完成当前的数据流。
启用任务命令格式:storm activate【拓扑名称】
storm activate topology-name
重新部署任务命令格式:storm rebalance 【拓扑名称】
storm rebalance topology-name
再平衡使你重分配集群任务。这是个很强大的命令。比如,你向一个运行中的集群增加了节点。再平衡命令将会停用拓扑,然后在相应超时时间之后重分配工人,并重启拓扑。

注意使用storm运行jar包的时候是没有输入输出路径的,区别于hadoop离线分析:

 [root@master storm]# bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.-incubating.jar storm.starter.WordCountTopology wordcount
Running: /home/hadoop/soft/jdk1..0_65/bin/java -client -Dstorm.options= -Dstorm.home=/home/hadoop/soft/apache-storm-0.9.-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-codec-1.6.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/hiccup-0.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/curator-client-2.4..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/clout-1.0..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/json-simple-1.1.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/httpclient-4.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/jgrapht-core-0.9..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/jline-2.11.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/ring-servlet-0.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/asm-4.0.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/clojure-1.5..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/joda-time-2.0.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/minlog-1.2.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/logback-classic-1.0..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/kryo-2.21.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/netty-3.6..Final.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/log4j-over-slf4j-1.6..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-lang-2.5.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/tools.logging-0.2..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-logging-1.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/compojure-1.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/logback-core-1.0..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/math.numeric-tower-0.0..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/ring-devel-0.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/servlet-api-2.5.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-fileupload-1.2..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/guava-13.0.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/jetty-6.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/disruptor-2.10..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/clj-time-0.4..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/jetty-util-6.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/httpcore-4.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/core.incubator-0.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/carbonite-1.4..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-io-2.4.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/clj-stacktrace-0.2..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/slf4j-api-1.6..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/curator-framework-2.4..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/netty-3.2..Final.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/ring-core-1.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/chill-java-0.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/commons-exec-1.1.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/tools.macro-0.1..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/storm-core-0.9.-incubating.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/ring-jetty-adapter-0.3..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/zookeeper-3.4..jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/servlet-api-2.5-.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/objenesis-1.2.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/lib/tools.cli-0.2..jar:examples/storm-starter/storm-starter-topologies-0.9.-incubating.jar:/home/hadoop/soft/apache-storm-0.9.-incubating/conf:/home/hadoop/soft/apache-storm-0.9.-incubating/bin -Dstorm.jar=examples/storm-starter/storm-starter-topologies-0.9.-incubating.jar storm.starter.WordCountTopology wordcount
[main] INFO backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
[main] INFO backtype.storm.StormSubmitter - Uploading topology jar examples/storm-starter/storm-starter-topologies-0.9.-incubating.jar to assigned location: storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar
Start uploading file 'examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar' to 'storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar' ( bytes)
[==================================================] /
File 'examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar' uploaded to 'storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar' ( bytes)
[main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar
[main] INFO backtype.storm.StormSubmitter - Submitting topology wordcount in distributed mode with conf {"topology.workers":,"topology.debug":true}
[main] INFO backtype.storm.StormSubmitter - Finished submitting topology: wordcount
[root@master storm]#

使用storm自带的统计的demo启动起来以后ui如下所示:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

然后点击wordCount以后如下所示,点击下面的按钮是相应的操作:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

点击上面的按钮是相应的操作;

出现如下错误,这里贴一下,出错原因是:/storm/conf/storm.yaml的配置nimbus.host: "master"前面多了一个空格,删除空格即可:

 [root@slaver1 bin]# ./storm supervisor
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.core$load_one.invoke(core.clj:)
at clojure.core$load_lib$fn__4967.invoke(core.clj:)
at clojure.core$load_lib.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$load_libs.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$use.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:)
at backtype.storm.command.config_value__init.load(Unknown Source)
at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.lang.Var.invoke(Var.java:)
at backtype.storm.command.config_value.<clinit>(Unknown Source)
Caused by: while parsing a block mapping
in 'reader', line , column :
storm.zookeeper.servers:
^
expected <block end>, but found BlockMappingStart
in 'reader', line , column :
nimbus.host: "master"
^ at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:)
at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:)
at backtype.storm.utils.Utils.readStormConfig(Utils.java:)
at backtype.storm.utils.Utils.<clinit>(Utils.java:)
... more
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.core$load_one.invoke(core.clj:)
at clojure.core$load_lib$fn__4967.invoke(core.clj:)
at clojure.core$load_lib.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$load_libs.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$use.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:)
at backtype.storm.command.config_value__init.load(Unknown Source)
at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.lang.Var.invoke(Var.java:)
at backtype.storm.command.config_value.<clinit>(Unknown Source)
Caused by: while parsing a block mapping
in 'reader', line , column :
storm.zookeeper.servers:
^
expected <block end>, but found BlockMappingStart
in 'reader', line , column :
nimbus.host: "master"
^ at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:)
at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:)
at backtype.storm.utils.Utils.readStormConfig(Utils.java:)
at backtype.storm.utils.Utils.<clinit>(Utils.java:)
... more
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at backtype.storm.config$loading__4910__auto__.invoke(config.clj:)
at backtype.storm.config__init.load(Unknown Source)
at backtype.storm.config__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.core$load_one.invoke(core.clj:)
at clojure.core$load_lib$fn__4967.invoke(core.clj:)
at clojure.core$load_lib.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$load_libs.doInvoke(core.clj:)
at clojure.lang.RestFn.applyTo(RestFn.java:)
at clojure.core$apply.invoke(core.clj:)
at clojure.core$use.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:)
at backtype.storm.command.config_value__init.load(Unknown Source)
at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:)
at clojure.lang.RT.loadClassForName(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.lang.RT.load(RT.java:)
at clojure.core$load$fn__5018.invoke(core.clj:)
at clojure.core$load.doInvoke(core.clj:)
at clojure.lang.RestFn.invoke(RestFn.java:)
at clojure.lang.Var.invoke(Var.java:)
at backtype.storm.command.config_value.<clinit>(Unknown Source)
Caused by: while parsing a block mapping
in 'reader', line , column :
storm.zookeeper.servers:
^
expected <block end>, but found BlockMappingStart
in 'reader', line , column :
nimbus.host: "master"
^ at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:)
at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:)
at backtype.storm.utils.Utils.readStormConfig(Utils.java:)
at backtype.storm.utils.Utils.<clinit>(Utils.java:)
... more

开始我用的apache-storm-0.9.2-incubating.tar.gz版本,使用如下官方demo,出现的问题是storm的ui的点击wordcount没有

Spouts (All time),Bolts (All time),Topology Visualization,Topology Configuration等等这些内容;

解决方法,我是换的新版本解决问题:

[root@slaver1 storm]# bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.6.jar storm.starter.WordCountTopology wordcount

6:在Storm的目录的bin目录下面,输入./storm命令可以查看Storm有何命令可使用:

#列出Topology计算拓补
[root@slaver1 bin]# ./storm list
#杀死Topology计算拓补
[root@slaver1 bin]# ./storm kill Topology这个计算拓补的名称

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

Storm的源码下载以及查看,点击Apache/storm即可进入Github进行源码下载:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

7:Idea创建的Maven项目打包,测试,如何操作呢???

最后打成的jar包就是target里面的jar包:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

运行如下所示:

 [root@master bin]# storm jar /home/hadoop/data_hadoop/storm-1.0-SNAPSHOT.jar com.bie.WordCountTopologyMain

效果如下所示:

一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

待续......