Is it possible manually compact prometheus wal block?

219 Views Asked by At

I have a test data that need to use to run tests. To generate this test data I've created prometheus + cadvisor containers and run some stress tests for 30 minutes. It generated me wal file with size of 2.8 mb. I need to keep this file for the later use, can I compact it somehow:

It is secured against crashes by a write-ahead log (WAL) that can be replayed when the Prometheus server restarts. Write-ahead log files are stored in the wal directory in 128MB segments. These files contain raw data that has not yet been compacted; thus they are significantly larger than regular block files.

If I understood correctly this file can be compacted to use much less space. Prometheus does this automatically after 2 hours, can I do it manually for custom files and then inject result into another prometheus to see the data?

1

There are 1 best solutions below

0
Anatoly On BEST ANSWER

Have created this script that seems to do the job:

package main

import (
    "fmt"
    "github.com/go-kit/log"
    "github.com/prometheus/prometheus/tsdb"
    "os"
)

func main() {
    dbPath := "./input_dir"
    logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stdout))
    db, err := tsdb.OpenDBReadOnly(dbPath, logger)
    if err != nil {
        fmt.Printf("Error opening DB: %v\n", err)
        return
    }
    err = db.FlushWAL("./output_dir")
    if err != nil {
        fmt.Printf("Error flushing WAL: %v\n", err)
        return
    }
}