Bitnami fluentd 部署,输出到ES出错

时间:2021-01-10 00:37:36

不加载ES相关的cm就没问题:

fluentd 04:34:48.31  

fluentd 04:34:48.31 Welcome to the Bitnami fluentd container

fluentd 04:34:48.31 Subscribe to project updates by watching https://github.com/bitnami/containers

fluentd 04:34:48.31 Submit issues and feature requests at https://github.com/bitnami/containers/issues

fluentd 04:34:48.31  

fluentd 04:34:48.31 INFO  ==> ** Starting Fluentd setup **

find: '/docker-entrypoint-initdb.d/': No such file or directory

fluentd 04:34:48.33 INFO  ==> No custom scripts in /docker-entrypoint-initdb.d

fluentd 04:34:48.33 INFO  ==> ** Fluentd setup finished! **


fluentd 04:34:48.34 INFO  ==> ** Starting Fluentd **

2023-03-02 04:34:50 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil

2023-03-02 04:34:50 +0000 [info]: parsing config file is succeeded path="/opt/bitnami/fluentd/conf/fluentd.conf"

2023-03-02 04:34:50 +0000 [info]: gem 'fluentd' version '1.15.3'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-concat' version '2.5.0'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.14'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.2.4'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-grafana-loki' version '1.2.20'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-kafka' version '0.18.1'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '3.1.3'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.0.3'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-s3' version '1.7.2'

2023-03-02 04:34:50 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'

2023-03-02 04:34:51 +0000 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead

2023-03-02 04:34:51 +0000 [info]: using configuration file: <ROOT>

 <match fluent.**>

   @type null

 </match>

 <source>

   @type forward

   bind "0.0.0.0"

   port 24224

 </source>

 <source>

   @type http

   bind "0.0.0.0"

   port 9880

 </source>

 <match fluentd.healthcheck>

   @type stdout

 </match>

 <match **>

   @type stdout

 </match>

</ROOT>

2023-03-02 04:34:51 +0000 [info]: starting fluentd-1.15.3 pid=1 ruby="3.1.3"

2023-03-02 04:34:51 +0000 [info]: spawn command to main:  cmdline=["/opt/bitnami/ruby/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/bitnami/fluentd/bin/fluentd", "--config", "/opt/bitnami/fluentd/conf/fluentd.conf", "--plugin", "/opt/bitnami/fluentd/plugins", "--under-supervisor"]

2023-03-02 04:34:51 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil

2023-03-02 04:34:53 +0000 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil

2023-03-02 04:34:53 +0000 [info]: adding match pattern="fluent.**" type="null"

2023-03-02 04:34:53 +0000 [info]: adding match pattern="fluentd.healthcheck" type="stdout"

2023-03-02 04:34:53 +0000 [info]: adding match pattern="**" type="stdout"

2023-03-02 04:34:53 +0000 [info]: adding source type="forward"

2023-03-02 04:34:53 +0000 [info]: adding source type="http"

2023-03-02 04:34:54 +0000 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead

2023-03-02 04:34:54 +0000 [info]: #0 starting fluentd worker pid=31 ppid=1 worker=0

2023-03-02 04:34:54 +0000 [info]: #0 listening port port=24224 bind="0.0.0.0"

2023-03-02 04:34:54 +0000 [info]: #0 fluentd worker is now running worker=0

加载了es相关的配置就出错:

helm install fluentd bitnami/fluentd -n efk-bitnami --version 5.5.15 \

 --set aggregator.configMap=elasticsearch-output \

 --set aggregator.extraEnv[0].name=ELASTICSEARCH_HOST \

 --set aggregator.extraEnv[0].value=elasticsearch-coordinating-hl.efk-bitnami.svc.cluster.local \

 --set aggregator.extraEnv[1].name=ELASTICSEARCH_PORT \

 --set aggregator.extraEnv[1].value=9200

从配置文件变了开始就出错:

2023-03-02 04:45:42 +0000 [info]: using configuration file: <ROOT>

 <source>

   @type prometheus

   port 24231

 </source>

 <source>

   @type prometheus_monitor

   <labels>

     host ${hostname}

   </labels>

 </source>

 <source>

   @type prometheus_output_monitor

   <labels>

     host ${hostname}

   </labels>

 </source>

 <match fluent.**>

   @type null

 </match>

 <source>

   @type forward

   bind "0.0.0.0"

   port 24224

 </source>

 <source>

   @type http

   bind "0.0.0.0"

   port 9880

 </source>

 <match fluentd.healthcheck>

   @type stdout

 </match>

 <match **>

   @type elasticsearch

   include_tag_key true

   host ""

   port tcp://172.19.30.234:9200

   logstash_format true

   <buffer>

     @type "file"

     path "/opt/bitnami/fluentd/logs/buffers/logs.buffer"

     flush_thread_count 2

     flush_interval 5s

   </buffer>

 </match>

</ROOT>

2023-03-02 04:45:42 +0000 [info]: starting fluentd-1.15.3 pid=1 ruby="3.1.3"

2023-03-02 04:45:42 +0000 [info]: spawn command to main:  cmdline=["/opt/bitnami/ruby/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/bitnami/fluentd/bin/fluentd", "--config", "/opt/bitnami/fluentd/conf/fluentd.conf", "--plugin", "/opt/bitnami/fluentd/plugins", "--under-supervisor"]

2023-03-02 04:45:42 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil

2023-03-02 04:45:44 +0000 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil

2023-03-02 04:45:44 +0000 [info]: adding match pattern="fluent.**" type="null"

2023-03-02 04:45:44 +0000 [info]: adding match pattern="fluentd.healthcheck" type="stdout"

2023-03-02 04:45:45 +0000 [info]: adding match pattern="**" type="elasticsearch"

The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.

2023-03-02 04:45:45 +0000 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `+' for nil:NilClass\n\n        datum[:headers]['Host']   ||= datum[:host] + port_string(datum)\n                                                   ^"

                                               ^"

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/excon-0.99.0/lib/excon/connection.rb:250:in `request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/faraday-excon-1.1.0/lib/faraday/adapter/excon.rb:31:in `block in call'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/faraday-1.10.3/lib/faraday/adapter.rb:50:in `connection'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/faraday-excon-1.1.0/lib/faraday/adapter/excon.rb:31:in `call'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/faraday-1.10.3/lib/faraday/rack_builder.rb:154:in `build_response'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/faraday-1.10.3/lib/faraday/connection.rb:516:in `run_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elastic-transport-8.2.0/lib/elastic/transport/transport/http/faraday.rb:41:in `block in perform_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elastic-transport-8.2.0/lib/elastic/transport/transport/base.rb:296:in `perform_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elastic-transport-8.2.0/lib/elastic/transport/transport/http/faraday.rb:36:in `perform_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elastic-transport-8.2.0/lib/elastic/transport/client.rb:176:in `perform_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elasticsearch-8.6.0/lib/elasticsearch.rb:160:in `elasticsearch_validation_request'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elasticsearch-8.6.0/lib/elasticsearch.rb:85:in `verify_elasticsearch'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elasticsearch-8.6.0/lib/elasticsearch.rb:70:in `method_missing'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/elasticsearch-api-8.6.0/lib/elasticsearch/api/actions/info.rb:41:in `info'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluent-plugin-elasticsearch-5.2.4/lib/fluent/plugin/out_elasticsearch.rb:496:in `detect_es_major_version'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluent-plugin-elasticsearch-5.2.4/lib/fluent/plugin/out_elasticsearch.rb:487:in `block in handle_last_seen_es_major_version'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluent-plugin-elasticsearch-5.2.4/lib/fluent/plugin/elasticsearch_index_template.rb:56:in `retry_operate'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluent-plugin-elasticsearch-5.2.4/lib/fluent/plugin/out_elasticsearch.rb:484:in `handle_last_seen_es_major_version'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluent-plugin-elasticsearch-5.2.4/lib/fluent/plugin/out_elasticsearch.rb:336:in `configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/plugin.rb:187:in `configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/agent.rb:132:in `add_match'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/agent.rb:74:in `block in configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/agent.rb:64:in `each'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/agent.rb:64:in `configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/root_agent.rb:149:in `configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/engine.rb:105:in `configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/engine.rb:80:in `run_configure'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/supervisor.rb:782:in `block in run_worker'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/supervisor.rb:1056:in `main_process'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/supervisor.rb:774:in `run_worker'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/lib/fluent/command/fluentd.rb:370:in `<top (required)>'

 2023-03-02 04:45:45 +0000 [error]: #0 <internal:/opt/bitnami/ruby/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require'

 2023-03-02 04:45:45 +0000 [error]: #0 <internal:/opt/bitnami/ruby/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/gems/fluentd-1.15.3/bin/fluentd:15:in `<top (required)>'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/bin/fluentd:25:in `load'

 2023-03-02 04:45:45 +0000 [error]: #0 /opt/bitnami/fluentd/bin/fluentd:25:in `<main>'

2023-03-02 04:45:46 +0000 [error]: Worker 0 exited unexpectedly with status 1