Logstash之filter：
json filter：
input{
stdin{
}
}
filter{
json{
source => "message"
}
}
output{
stdout{
codec => json
}
输入：
{"name": "CSL", "age": 20}
输出：
Grok filter:
pattern：
https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns
创建一个测试log：
[sky@hadoop1 bin]$ cat spark-test-log.log
05/30/17 17:13:24 INFO StartingSparkmasteratspark
05/30/17 17:13:24 INFO RunningSparkversion1
05/30/17 17:13:25 INFO jetty
创建conf：
input{
file{
path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"
type => "sparkfile"
start_position => "beginning"
}
}
filter{
grok{
match => ["message", "%{DATE:date} %{TIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]
}
}
output{
stdout{
codec => rubydebug
}
}
运行结果：
自定义正则表达式：
[sky@hadoop1 patterns]$ cat selfpattern
SKYTIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])
修改conf:
input{
file{
path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"
type => "sparkfile"
start_position => "beginning"
}
}
filter{
grok{
patterns_dir => '/usr/local/logstash-5.6.1/patterns/selfpattern'
match => ["message", "%{DATE:date} %{SKYTIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]
}
}
output{
stdout{
codec => rubydebug
}
}
输出结果：
定义多个match：使用，分隔。
测试正则表达式网址：