Logstash filter - mask secrets in json data / replace specific keys values

Question

I have some JSON data sent in to my logstash filter and wish to mask secrets from appearing in Kibana. My log looks like this:

{
    "payloads":
    [
        {
            "sequence": 1,
            "request":
            {
                "url": "https://hello.com",
                "method": "POST",
                "postData": "{\"one:\"1\",\"secret:"THISISSECRET",\"username\":\"hello\",\"secret2\":\"THISISALSOSECRET\"}",
            },
            "response":
            {
                "status": 200,
            }
        }
    ],
...

My filter converts the payloads to payload and I then wish to mask the JSON in postData to be:

"postData": "{\"one:\"1\",\"secret\":\"[secret]\",\"username\":\"hello\",\"secret2\":\"[secret]\"}"

My filter now looks like this:

if ([payloads]) {
  split {
    field => "payloads"
    target => "payload"
    remove_field => [payloads]
  }
}

# innetTmp is set to JSON here - this works

json {
  source => "innerTmp"
  target => "parsedJson"
  if [parsedJson][secret] =~ /.+/ {
    remove_field => [ "secret" ]
    add_field => { "secret" => "[secret]" }
  }
  if [parsedJson][secret2] =~ /.+/   {
    remove_field => [ "secret2" ]
    add_field => { "secret2" => "[secret]" }
  }
}

Is this a correct approach? I cannot see the filter replacing my JSON key/values with "[secret]".

Kind regards /K

Answer 1

The approach is good, you are using the wrong field

After the split the secret field is part of postData and that field is part of parsedJson.

  if [parsedJson][postData][secret]  {
    remove_field => [ "[parsedJson][postData][secret]" ]
    add_field => { "[parsedJson][postData][secret]" => "[secret]" }
  }