Flume 1.9.0 的安装(比较简单, 操作也不像老版本那么繁琐了)

时间：2020-12-12 03:02:21

之前已经完成了Hadoop集群、Hbase集群、Hive的搭建, 这次来安装一下flume-1.9.0

安装过程

  1. 将tar包上传并解压到指定目录, 并修改名称

    tar -zxvf apache-flume-1.9.0-bin.tar.gz -C /opt/ronnie
    
cd /opt/ronnie/
    
mv apache-flume-1.9.0-bin/ flume-1.9.0

  2. 进入flume配置文件目录, 拷贝一份环境配置文件并修改

    cd flume-1.9.0/conf/
    
cp flume-env.sh.template flume-env.sh
    
vim flume-env.sh
    # 修改JAVA_HOME
    
export JAVA_HOME=/usr/lib/jvm/jdk1.8

  3. 配置环境变量

    • vim ~/.bashrc

      # Flume
      
export FLUME_HOME=/opt/ronnie/flume-1.9.0
      
export PATH=$FLUME_HOME/bin:$PATH

    • source ~/.bashrc

  4. 查看flume版本

    flume-ng version

# result
    
Flume 1.9.0
    
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
    
Revision: d4fcab4f501d41597bc616921329a4339f73585e
    
Compiled by fszabo on Mon Dec 17 20:45:25 CET 2018
    
From source with checksum 35db629a3bda49d23e9b3690c80737f9

  5. 启动flume

    • 配置flume运行文件flume-conf.properties

      cp flume-conf.properties.template flume-conf.properties
      
vim flume-conf.properties

      
# 1.Name the components on this agent
      
a1.sources = r1
      
a1.sinks = k1
      
a1.channels = c1

# 2.Describe/configure the source
      
a1.sources.r1.type = netcat
      
a1.sources.r1.bind = node02 # 我改了hostname文件
      
a1.sources.r1.port = 44444

# 3.Describe the sink
      
a1.sinks.k1.type = logger

# 4.Use a channel which buffers events in memory
      
a1.channels.c1.type = memory
      
a1.channels.c1.capacity = 1000
      
a1.channels.c1.transactionCapacity = 100

# 5.Bind the source and sink to the channel
      
a1.sources.r1.channels = c1
      
a1.sinks.k1.channel = c1

    • 运行flume

      flume-ng agent \
      
--name a1 \
      
--conf $FLUME_HOME/conf \
      
--conf-file $FLUME_HOME/conf/flume-conf.properties \
      
-Dflume.root.logger=INFO,console

    • 启动成功

      erGroup.java:119)] Monitored counter group for type: CHANNEL, name: c1: Successfully registered new MBean.
      
2019-12-09 16:05:05,927 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)] Component type: CHANNEL, name: c1 started
      
2019-12-09 16:05:05,927 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:196)] Starting Sink k1
      
2019-12-09 16:05:05,927 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:207)] Starting Source r1
      
2019-12-09 16:05:05,928 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:155)] Source starting
      
2019-12-09 16:05:05,935 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:166)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/192.168.180.131:44444]

相关文章