How to send haproxy logs to fluentd by td-agent?

Question

I wanna send haproxy logs to fluentd/elasticsearch/kibana using td-agent, but I can't do it correctly

I have installed EFK by dockers and it rules correctly. I have a haproxy with log type haproxy.tcp like this:

haproxy[27508]: info 127.0.0.1:45111 [12/Jul/2012:15:19:03.258] wss-relay wss-relay/local02_9876 0/0/50015 1277 cD 1/0/0/0/0 0/0

My td-agent.conf is this:

<source>
  @type tail
  path /var/log/haproxy.log
  format /^(?<ps>\w+)\[(?<pid>\d+)\]: (?<pri>\w+) (?<c_ip>[\w\.]+):(?<c_port>\d+) \[(?<time>.+)\] (?<f_end>[\w-]+) (?<b_end>[\w-]+)\/(?<b_server>[\w-]+) (?<tw>\d+)\/(?<tc>\d+)\/(?<tt>\d+) (?<bytes>\d+) (?<t_state>[\w-]+) (?<actconn>\d+)\/(?<feconn>\d+)\/(?<beconn>\d+)\/(?<srv_conn>\d+)\/(?<retries>\d+) (?<srv_queue>\d+)\/(?<backend_queue>\d+)$/
  tag haproxy.tcp
  time_format %d/%B/%Y:%H:%M:%S
</source>
<match haproxy.tcp>
  @type forward
  <server>
    host dockerdes01
    port 24224
  </server>
</match>

But the log don't arrive to /var/log/td-agent/td-agent.log

If I use this :

<match haproxy.tcp>
  @type copy
  <store>
    @type stdout
  </store>
  <store>
    @type elasticsearch
    logstash_format true
    flush_interval 10s # for testing.
    host dockerdes01
    port 9200
  </store>
</match>

I see this in my /var/log/td-agent/td-agent.log:

2012-07-12 15:19:03.000000000 +0200 haproxy.tcp: {"ps":"haproxy","pid":"27508","pri":"info","c_ip":"127.0.0.1","c_port":"45111","f_end":"wss-relay","b_end":"wss-relay","b_server":"local02_9876","tw":"0","tc":"0","tt":"50015","bytes":"1277","t_state":"cD","actconn":"1","feconn":"0","beconn":"0","srv_conn":"0","retries":"0","srv_queue":"0","backend_queue":"0"}

but it doesn't arrive to fluentd...

I need that the logs arrive to fluentd

Answer 1

Better to use syslog set up to fluentd and just send from haproxy with syslog.