在kafka单节点测试的基础上
1.进行单节点多代理配置
新建3个目录用于存放kafka数据
lcc@localhost config$ mkdir /Users/lcc/soft/kafka/datas/logs_1
lcc@localhost config$ mkdir /Users/lcc/soft/kafka/datas/logs_2
lcc@localhost config$ mkdir /Users/lcc/soft/kafka/datas/logs_3复制3分配置文件
lcc@localhost config$ cp /Users/lcc/soft/kafka/kafka/config/server.properties /Users/lcc/soft/kafka/kafka/config/server-1.properties
lcc@localhost config$ cp /Users/lcc/soft/kafka/kafka/config/server.properties /Users/lcc/soft/kafka/kafka/config/server-2.properties
lcc@localhost config$ cp /Users/lcc/soft/kafka/kafka/config/server.properties /Users/lcc/soft/kafka/kafka/config/server-3.properties
lcc@localhost config$ rm -rf /Users/lcc/soft/kafka/kafka/config/server.properties配置3粉文件
lcc@localhost config$ vim /Users/lcc/soft/kafka/kafka/config/server-1.properties
broker.id=0
port=9092
log.dirs=/Users/lcc/soft/kafka/datas/logs_1
lcc@localhost config$ vim /Users/lcc/soft/kafka/kafka/config/server-2.properties
broker.id=1
port=9093
log.dirs=/Users/lcc/soft/kafka/datas/logs_2
lcc@localhost config$ vim /Users/lcc/soft/kafka/kafka/config/server-3.properties
broker.id=2
port=9094
log.dirs=/Users/lcc/soft/kafka/datas/logs_3启动多个代理 - 在三台服务器上进行所有更改后，打开三个新终端，逐个启动每个代理。
lcc@localhost bin$ /Users/lcc/soft/kafka/kafka/bin/kafka-server-start.sh /Users/lcc/soft/kafka/kafka/config/server-1.properties
lcc@localhost bin$ /Users/lcc/soft/kafka/kafka/bin/kafka-server-start.sh /Users/lcc/soft/kafka/kafka/config/server-2.properties
lcc@localhost bin$ /Users/lcc/soft/kafka/kafka/bin/kafka-server-start.sh /Users/lcc/soft/kafka/kafka/config/server-3.properties可以看到启动成功
lcc@localhost config$ jps
26657 Kafka
8961 RunJar
21252 RemoteMavenServer
8229 ResourceManager
8581 DataNode
26410 Kafka
8906 RunJar
21739 QuorumPeerMain
27410 Jps
8116 SecondaryNameNode
26901 Kafka
21240
21753 QuorumPeerMain
21724 QuorumPeerMain
7935 NameNode
lcc@localhost config$查看log_N数据
lcc@localhost config$ ll /Users/lcc/soft/kafka/datas/logs_1
total 32
drwxr-xr-x 59 lcc staff 2006 6 8 14:48 ./
drwxr-xr-x 5 lcc staff 170 6 8 14:35 ../
-rw-r--r-- 1 lcc staff 0 6 8 14:46 .lock
drwxr-xr-x 6 lcc staff 204 6 8 14:46 __consumer_offsets-0/
drwxr-xr-x 6 lcc staff 204 6 8 14:46 __consumer_offsets-1/
-rw-r--r-- 1 lcc staff 0 6 8 14:46 cleaner-offset-checkpoint
-rw-r--r-- 1 lcc staff 4 6 8 14:47 log-start-offset-checkpoint
-rw-r--r-- 1 lcc staff 54 6 8 14:46 meta.properties
-rw-r--r-- 1 lcc staff 1204 6 8 14:47 recovery-point-offset-checkpoint
-rw-r--r-- 1 lcc staff 1204 6 8 14:48 replication-offset-checkpoint
drwxr-xr-x 6 lcc staff 204 6 8 14:46 xtwy-0/
lcc@localhost config$ ll /Users/lcc/soft/kafka/datas/logs_2
total 8
drwxr-xr-x 8 lcc staff 272 6 8 14:46 ./
drwxr-xr-x 5 lcc staff 170 6 8 14:35 ../
-rw-r--r-- 1 lcc staff 0 6 8 14:46 .lock
-rw-r--r-- 1 lcc staff 0 6 8 14:46 cleaner-offset-checkpoint
-rw-r--r-- 1 lcc staff 0 6 8 14:46 log-start-offset-checkpoint
-rw-r--r-- 1 lcc staff 54 6 8 14:46 meta.properties
-rw-r--r-- 1 lcc staff 0 6 8 14:46 recovery-point-offset-checkpoint
-rw-r--r-- 1 lcc staff 0 6 8 14:46 replication-offset-checkpoint
lcc@localhost config$ ll /Users/lcc/soft/kafka/datas/logs_3
total 8
drwxr-xr-x 8 lcc staff 272 6 8 14:47 ./
drwxr-xr-x 5 lcc staff 170 6 8 14:35 ../
-rw-r--r-- 1 lcc staff 0 6 8 14:47 .lock
-rw-r--r-- 1 lcc staff 0 6 8 14:47 cleaner-offset-checkpoint
-rw-r--r-- 1 lcc staff 0 6 8 14:47 log-start-offset-checkpoint
-rw-r--r-- 1 lcc staff 54 6 8 14:47 meta.properties
-rw-r--r-- 1 lcc staff 0 6 8 14:47 recovery-point-offset-checkpoint
-rw-r--r-- 1 lcc staff 0 6 8 14:47 replication-offset-checkpoint
lcc@localhost config$我们来逐步看看数据
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/
.lock __consumer_offsets-21/ __consumer_offsets-35/ __consumer_offsets-49/
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/.lock
connect-console-sink.properties connect-file-source.properties log4j.properties server-3.properties
connect-console-source.properties connect-log4j.properties producer.properties tools-log4j.properties
connect-distributed.properties connect-standalone.properties server-1.properties zookeeper.properties
connect-file-sink.properties consumer.properties server-2.properties
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/.lock.lock文件中，基本上都是配置信息
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/__consumer_offsets-1/
00000000000000000000.index 00000000000000000000.log 00000000000000000000.timeindex leader-epoch-checkpoint
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/__consumer_offsets-1/00000000000000000000.index
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@一堆我们看不懂的文件，
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/__consumer_offsets-1/00000000000000000000.timeindex这个文件也是看不懂的，leader-epoch-checkpoint文件里面没数据
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/meta.properties
#Fri Jun 08 14:46:07 CST 2018
version=0
broker.id=0lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/recovery-point-offset-checkpoint
0
51
__consumer_offsets 22 0
__consumer_offsets 30 0
__consumer_offsets 35 0
xtwy 0 0
__consumer_offsets 41 0
__consumer_offsets 42 0lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/replication-offset-checkpoint
51
__consumer_offsets 22 0
__consumer_offsets 35 0
xtwy 0 0
__consumer_offsets 41 0
__consumer_offsets 36 0lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/xtwy-0/
00000000000000000000.index 00000000000000000000.log 00000000000000000000.timeindex leader-epoch-checkpoint
lcc@localhost config$ vim /Users/lcc/soft/kafka/datas/logs_1/xtwy-0/创建主题
让我们为此主题将复制因子值指定为三个，因为我们有三个不同的代理运行。 如果您有两个代理，那么分配的副本值将是两个。
lcc@localhost config$ ../bin/kafka-topics.sh --create --zookeeper localhost:2181,localhost:2182,localhost:2183 --replication-factor 3 -partitions 1 --topic topip-lcc
Created topic "topip-lcc".
lcc@localhost config$Describe 命令用于检查哪个代理正在侦听当前创建的主题，如下所示 -
lcc@localhost kafka$ bin/kafka-topics.sh --describe --zookeeper localhost:2181,localhost:2182,localhost:2183 --topic topip-lcc
Topic:topip-lcc PartitionCount:1 ReplicationFactor:3 Configs:
Topic: topip-lcc Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
lcc@localhost kafka$从上面的输出，我们可以得出结论，第一行给出所有分区的摘要，显示主题名称，分区数量和我们已经选择的复制因子。 在第二行中，每个节点将是分区的随机选择部分的领导者。
在我们的例子中，我们看到我们的第2个broker(Leader: 2 的意思是配置文件中broker.id=2)是领导者。 后面的暂时不会。
启动生产者以发送消息
此过程保持与单代理设置中相同。
lcc@localhost kafka$ bin/kafka-console-producer.sh --broker-list localhost:9092,localhost:9093,localhost:9094 --topic topip-lcc
>lss
>sdjfbhgsj
>dsjbfhhjsd
>启动消费者以接收消息
此过程保持与单代理设置中所示的相同。
lcc@localhost kafka$ ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topip-lcc --from-beginning
lss
sdjfbhgsj
dsjbfhhjsd