td-agent 收集日志到kafka的配置

时间:2024-07-22 14:05:02

<source>
@type tail
path /data1/td-agent/wechat-tag*.log
tag wechat-tag
read_from_head true
format none
pos_file /var/log/td-agent/wechat-tag.pos
</source>

<match wechat-tag>
@type kafka_buffered
brokers myserver.com:9092
buffer_type file
buffer_path /var/log/td-agent/buffer/wechat-tag
flush_interval 10s
output_data_type attr: message       
</match>

解释说明:

topic: wechat-tag

broker:myserver.com:9092

pos_file 为了记录消费文件的位置,防止重启tdagent以后重复消费

output_data_type:只发送message字段里的内容