How to use FluentD as a buffer between Telegraf and InfluxDB

Question

Is there any way to ship metrics gathered form Telegraf to FluentD, then into InfluxDB?

I know it's possible to write data from FluentD into InfluxDB; but how does one ship data from Telegraf into FluentD, basically using use FluentD as a buffer (as opposed to using Kafka or Redis)?

Answer 1

While it might be possible to do with FluentD using some of the available, although outdated output plugins, such as InfluxDB-Metrics, I couldn't get the plugin to work properly and it hasn't been updated in over six years, so it will probably not work with newer releases of FluentD.

Fluent Bit however, has an Influxdb output built right into it, so I was able to get it to work with that. The caveat is that it has no Telegraf plugin. So the solution I found was to setup a tcp input plugin in Fluent Bit, and set Telegraf to write JSON formatted data to it in it's output section.

The caveat of doing this, is that the JSON data is nested and not formatted properly for InfluxDB. The workaround is to use nest filters in Fluent Bit to 'lift' the nested data format, and re-format properly for InfluxDB.

Below is an example for disk-space, which is not a metric that is natively supported with Fluent Bit metrics but is natively supported with Telegraf:

@SET me=${HOST_HOSTNAME}

[INPUT]  ## tcp recipe  ## Collect data from telegraf
    Name          tcp
    Listen        0.0.0.0
    Port          5170
    Tag           telegraf.${me}
    Chunk_Size    32
    Buffer_Size   64
    Format        json
[FILTER]  ## rename the three tags sent from Telegraf to prevent duplicates
    Name          modify
    Match         telegraf.*
    Condition Key_Value_Equals name disk
    Rename        fields   fieldsDisk
    Rename        name     nameDisk
    Rename        tags     tagsDisk
[FILTER]  ## un-nest nested JSON formatted info under 'field' tag
    Name          nest
    Match         telegraf.*
    Operation     lift
    Nested_under  fieldsDisk
    Add_prefix    disk.
[FILTER]  ## un-nest nested JSON formatted info under 'disk' tag
    Name          nest
    Match         telegraf.*
    Operation     lift
    Nested_under  tagsDisk
    Add_prefix    disk.
[OUTPUT]  ## output properly formatted JSON info
    Name          influxdb
    Match         telegraf.*
    Host          influxdb.server.com
    Port          8086
    #HTTP_User     whatever
    #HTTP_Passwd   whatever
    Database      telegraf.${me}
    Sequence_Tag  point_in_time
    Auto_Tags     On

NOTE: This is just a simple awkward config for my own proof of concept