Logstash to Google cloud storage bucket is not working

55 Views Asked by At

I have one requirement to run a logstash pipeline which should send the logs to Google storage cloud. It should rotate the log for every one hour and send to GCP bucket.Also my main requirement is, the logs should save in a folder and date folder should dynamically create.

AWS s3 outputs are working fine for me, need the configuarion for GCP bucket. Anyone can you please share your logstash configuration file.

For testing purpose, I tried by using file input and output with 30 seconds. Please correct me if I am wrong.

input {
  file {
    path => "/tmp/*.log"
    type => "log"
  }
}

output {
  google_cloud_storage {
    bucket => "gcp-bucket/log_folder/%Y-%m-%d"
    date_pattern => "%Y-%m-%dT%S:30"
    uploader_interval_secs => 30
    log_file_prefix => "test"
    gzip_content_encoding => true
    gzip => true
    include_hostname => true
    include_uuid => true
    output_format => json
    temp_directory => "/tmp/logstash-gcs"
  }
}

0

There are 0 best solutions below