Send logs directly to Loki without use of agents

Question

Is there a way to send logs to Loki directly without having to use one of it's agents?

For example, if I have an API, is it possible to send request/response logs directly to Loki from an API, without the interference of, for example, Promtail?

Answer 1

Loki HTTP API

Loki HTTP API allows pushing messages directly to Grafana Loki server:

POST /loki/api/v1/push

/loki/api/v1/push is the endpoint used to send log entries to Loki. The default behavior is for the POST body to be a snappy-compressed protobuf message:

• Protobuf definition
• Go client library

Alternatively, if the Content-Type header is set to application/json, a JSON post body can be sent in the following format:

{
  "streams": [
    {
      "stream": {
        "label": "value"
      },
      "values": [
          [ "<unix epoch in nanoseconds>", "<log line>" ],
          [ "<unix epoch in nanoseconds>", "<log line>" ]
      ]
    }
  ]
}

You can set Content-Encoding: gzip request header and post gzipped JSON.

Example:

curl -v -H "Content-Type: application/json" -XPOST -s "http://localhost:3100/loki/api/v1/push" --data-raw \
 '{"streams": [{ "stream": { "foo": "bar2" }, "values": [ [ "1570818238000000000", "fizzbuzz" ] ] }]}'

So it is easy to create JSON-formatted string with logs and send it to the Grafana Loki.

Libraries

There are some libraries implementing several Grafana Loki protocols.

There is also (my) zero-dependency library in pure Java 1.8, which implements pushing logs in JSON format to Grafana Loki. Works on Java SE and Android platform:

• https://github.com/mjfryc/mjaron-tinyloki-java

Security

Above API doesn't support any access restrictions as written here - when using over public network, consider e.g. configuring Nginx proxy with HTTPS from Certbot and Basic Authentication.

Answer 2

Yes. You can send logs directly from a Java application to loki.

It can be done using the loki4j configuration in your java springboot project. Add these below dependencies to pom.xml

    <dependency>
        <groupId>com.github.loki4j</groupId>
        <artifactId>loki-logback-appender</artifactId>
        <version>1.2.0</version>
    </dependency>

Run loki either directly or from docker depending on how you have installed loki on your system. I use docker instances of loki and grafana.

Create a logback.xml in your springboot project with the following contents

    <property name="HOME_LOG" value="app.log" />

    <appender name="FILE-ROLLING"
        class="com.github.loki4j.logback.Loki4jAppender">
        <http>
            <url>http://localhost:3100/loki/api/v1/push</url>
        </http>
        <format>
            <label>
                <pattern>app=my-app,host=${HOSTNAME},level=%level</pattern>
            </label>
            <message>
                <pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex
                </pattern>
            </message>
            <sortByTime>true</sortByTime>
        </format>
    </appender>

    <logger name="com.vasanth.loki" level="debug" additivity="false">
        <appender-ref ref="FILE-ROLLING" />
    </logger>

    <root level="error">
        <appender-ref ref="FILE-ROLLING" />
    </root>

</configuration>

Configure your logger names in the above example and make sure you have given the proper loki URL - You are basically telling the application to write logs into an output stream going directly to the loki URL instead of the traditional way of writing logs to a file through log4j configuration and then using promtail to fetch these logs and load into loki.

Answer 3

Logs can be pushed to Loki in JSON or compresses protobuf format via /loki/api/v1/push. Note that it isn't recommended storing log messages with labels containing big number of unique values (such as trace_id, user_id, ip, etc.), since this may lead to high memory usage and slowdown for Grafana Loki (aka high cardinality issues). See these docs for details.