Flume接收Log4j日志

时间:2020-12-28 21:49:01

注意依赖一定要有flume-ng-log4jappender.jar,后面配置log4j要用。  

log4j,properties配置如下:

<span style="font-size:14px;">### set log levels ###
log4j.rootLogger=INFO, stdout, file, flume
log4j.logger.per.flume=INFO

### flume ###
log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n
log4j.appender.flume.Hostname=localhost
log4j.appender.flume.Port=44446

### stdout ###
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Threshold=INFO
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n

### file ###
log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.Threshold=INFO
log4j.appender.file.File=./logs/tracker/tracker.log
log4j.appender.file.Append=true
log4j.appender.file.DatePattern='.'yyyy-MM-dd
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n</span>


log4j.xml配置如下:

flume单机:

<span style="font-size:14px;"><appender name="flume-avro" class="org.apache.flume.clients.log4jappender.Log4jAppender">
<param name="Hostname" value="localhost" />
<param name="Port" value="44444" />
<param name="UnsafeMode" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%l] [%rms] %m" />
</layout>
</appender></span>


flume多节点:
<span style="font-size:14px;"><appender name="flume-load-balancing-avro" class="org.apache.flume.clients.log4jappender.LoadBalancingLog4jAppender">
<param name="Hosts"
value="xxx:12306 xxx:12306 xxx:12306" />
<param name="Selector" value="ROUND_ROBIN" />
<param name="MaxBackoff" value="2000" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} [%l] [%rms] %m" />
</layout>
</appender></span>

<appender name="flume-async" class="org.apache.log4j.AsyncAppender">
<param name="BufferSize" value="256" />
<appender-ref ref="flume-load-balancing-avro" />
</appender>