ZonedDateTime format- Elasticsearch index mapping

Question

I am trying to specify the format of my date in my Elasticsearch index mapping like this documentation states: https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

I have a date in this format: 2020-10-29T05:36:06.143Z[UTC]. How would I turn this into a format like YYYY-MM-DD...? Specifically, how to represent the [UTC] part at the end?

Currently I have the following mapping, I'm getting the error below, so I am trying to specify the format of the date to see if that works. Thanks!

    "createdTimeStamp": {
      "type": "date"
    },

Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [createdTimeStamp] of type [date] in document with id 'testId1'.

Preview of field's value: '{offset={totalSeconds=0, rules={fixedOffset=true, transitionRules=[], transitions=[]}, id=Z}, year=2015, dayOfYear=1, nano=0, chronology={calendarType=iso8601, id=ISO}, minute=10, second=30, dayOfWeek=THURSDAY, month=JANUARY, hour=12, zone={totalSeconds=0, rules={fixedOffset=true, transitionRules=[], transitions=[]}, id=Z}, dayOfMonth=1, monthValue=1}']];

nested: ElasticsearchException[Elasticsearch exception [type=illegal_state_exception, reason=Can't get text on a START_OBJECT at 1:72]]

Answer 1

You'll first need to escape Z itself -- it's a symbol that represents zone offsets and is not a valid value in and of itself.

PUT myindex/
{
  "mappings": {
    "properties": {
      "createdTimeStamp": {
        "type": "date",
        "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z[UTC]'"
      }
    }
  }
}

If you know for sure that all the incoming values will be in UTC, you can leave the above as-is. This'll escape that [UTC] part too.

If you expect multiple time zones, you can use yyyy-MM-dd'T'HH:mm:ss.SSS'Z['z']' instead -- the lowercase z will parse the incoming time zone for you and store the date internally in UTC but with an offset. That way, you'll only have escaped the uppercase Z for the reasons described above.

Answer 2

I do not have expertise in Elasticsearch. However, the following solution should help you resolve your issue.

You can mention the format, uuuu-MM-dd'T'HH:mm:ss.SSSX['['z']'] as shown in the example on the documentation page.

Demo:

import java.time.OffsetDateTime;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;

public class Main {
    public static void main(String[] args) {
        String strDateTime = "2020-10-29T05:36:06.143Z[UTC]";
        DateTimeFormatter formatter = DateTimeFormatter.ofPattern("uuuu-MM-dd'T'HH:mm:ss.SSSX['['z']']");

        ZonedDateTime zdt = ZonedDateTime.parse(strDateTime, formatter);
        System.out.println(zdt);

        OffsetDateTime odt = OffsetDateTime.parse(strDateTime, formatter);
        System.out.println(odt);
    }
}

Output:

2020-10-29T05:36:06.143Z[UTC]
2020-10-29T05:36:06.143Z

The ['['z']'] at the end of the format makes '['z']' optional where [ and ] are literals while z specifies the time-zone.

Some useful information:

Your date-time string has Z[UTC] at the end where Z is an important letter specifying Zulu date-time which is nothing but UTC date-time. In other words, your date-time represents a date-time in UTC. The ZonedDateTime can parse it without requiring any formatter.

Demo:

import java.time.ZonedDateTime;

public class Main {
    public static void main(String[] args) {
        String strDateTime = "2020-10-29T05:36:06.143Z[UTC]";
        ZonedDateTime zdt = ZonedDateTime.parse(strDateTime);
        System.out.println(zdt);
    }
}

Output:

2020-10-29T05:36:06.143Z[UTC]

However, it looks like Elasticsearch does not use ZonedDateTime to parse the date-time strings. The documentation page mentions that the default format is either strict_date_optional_time or epoch_millis as quoted below:

Date formats can be customised, but if no format is specified then it uses the default:

"strict_date_optional_time||epoch_millis"

So, in order to conform to the default format, an alternative way is to get rid of [UTC] from the end of your date-time string. After this change, the string can be parsed by all of ZonedDateTime, OffsetDateTime, and Instant without requiring any formatter.

Demo:

import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.ZonedDateTime;

public class Main {
    public static void main(String[] args) {
        String strDateTime = "2020-10-29T05:36:06.143Z[UTC]";
        strDateTime = strDateTime.substring(0, strDateTime.indexOf('['));
        System.out.println("Trimmed date-time string: " + strDateTime);

        ZonedDateTime zdt = ZonedDateTime.parse(strDateTime);
        System.out.println(zdt);

        OffsetDateTime odt = OffsetDateTime.parse(strDateTime);
        System.out.println(odt);

        Instant instant = Instant.parse(strDateTime);
        System.out.println(instant);
    }
}

Output:

Trimmed date-time string: 2020-10-29T05:36:06.143Z
2020-10-29T05:36:06.143Z
2020-10-29T05:36:06.143Z
2020-10-29T05:36:06.143Z