Gson @AutoValue and Optional<> dont work together, is there a workaround?

Question

Gson doesnt have direct support for serializing @AutoValue classes or for Optional<> fields, but com.ryanharter.auto.value adds @AutoValue and net.dongliu:gson-java8-datatype adds Optional<> and other java8 types.

However, they dont work together AFAICT.

Test code:

public class TestOptionalWithAutoValue {
  private static final Gson gson = new GsonBuilder().serializeNulls()
          // doesnt matter which order these are registered in
          .registerTypeAdapterFactory(new GsonJava8TypeAdapterFactory())
          .registerTypeAdapterFactory(AutoValueGsonTypeAdapterFactory.create())
          .create();

  @Test
  public void testAutoValueOptionalEmpty() {
    AvoTestClass subject = AvoTestClass.create(Optional.empty());

    String json = gson.toJson(subject, AvoTestClass.class);
    System.out.printf("Json produced = %s%n", json);
    AvoTestClass back = gson.fromJson(json, new TypeToken<AvoTestClass>() {}.getType());
    assertThat(back).isEqualTo(subject);
  }

  @Test
  public void testAutoValueOptionalFull() {
    AvoTestClass subject = AvoTestClass.create(Optional.of("ok"));

    String json = gson.toJson(subject, AvoTestClass.class);
    System.out.printf("Json produced = '%s'%n", json);
    AvoTestClass back = gson.fromJson(json, new TypeToken<AvoTestClass>() {}.getType());
    assertThat(back).isEqualTo(subject);
  }
}

@AutoValue
public abstract class AvoTestClass {
  abstract Optional<String> sval();

  public static AvoTestClass create(Optional<String> sval) {
    return new AutoValue_AvoTestClass(sval);
  }

  public static TypeAdapter<AvoTestClass> typeAdapter(Gson gson) {
    return new AutoValue_AvoTestClass.GsonTypeAdapter(gson);
  }
}

@GsonTypeAdapterFactory
public abstract class AutoValueGsonTypeAdapterFactory implements TypeAdapterFactory {
  public static TypeAdapterFactory create() {
    return new AutoValueGson_AutoValueGsonTypeAdapterFactory();
  }
}

gradle dependencies:

    annotationProcessor "com.google.auto.value:auto-value:1.7.4"
    annotationProcessor("com.ryanharter.auto.value:auto-value-gson-extension:1.3.1")
    implementation("com.ryanharter.auto.value:auto-value-gson-runtime:1.3.1")
    annotationProcessor("com.ryanharter.auto.value:auto-value-gson-factory:1.3.1")

    implementation 'net.dongliu:gson-java8-datatype:1.1.0'

Fails with:

Json produced = {"sval":null}
...
java.lang.NullPointerException: Null sval
...

net.dongliu.gson.OptionalAdapter is called on serialization, but not deserialization.

Im wondering if theres a workaround, or if the answer is that Gson needs to have direct support for Optional<> ?

Answer 1

Glad to see you've updated your question by adding much more information and even by adding a test! :) That really makes it clear!

I'm not sure, but the generated type adapter has no mention for the default value for sval:

jsonReader.beginObject();
// [NOTE] This is where it is initialized with null, so I guess it will definitely fail if the `sval` property is not even present in the deserialized JSON object
Optional<String> sval = null;
while (jsonReader.hasNext()) {
String _name = jsonReader.nextName();
// [NOTE] This is where it skips `null` value so it even does not reach to the `OptionalAdapter` run
if (jsonReader.peek() == JsonToken.NULL) {
  jsonReader.nextNull();
  continue;
}
switch (_name) {
  default: {
    if ("sval".equals(_name)) {
      TypeAdapter<Optional<String>> optional__string_adapter = this.optional__string_adapter;
      if (optional__string_adapter == null) {
        this.optional__string_adapter = optional__string_adapter = (TypeAdapter<Optional<String>>) gson.getAdapter(TypeToken.getParameterized(Optional.class, String.class));
      }
      sval = optional__string_adapter.read(jsonReader);
      continue;
    }
    jsonReader.skipValue();
  }
}
}
jsonReader.endObject();
return new AutoValue_AvoTestClass(sval);

I have no idea if there is a way to configure the default values in AutoValue or other generators you mentioned, but it looks like a bug.

If there is no any way to work around it (say, library development abandoned; it takes too much time to wait for a fix; whatever), you can always implement it yourself, however with some runtime cost (basically this how Gson works under the hood for data bag objects). The idea is delegating the job to the built-in RuntimeTypeAdapterFactory so that it could deal with a concrete class, not an abstract one, and set all fields according to the registered type adapters (so that the Java 8 types are supported as well). The cost here is reflection, thus that adapter may work slower than generated type adapters. Another thing is that if a JSON property does not even encounter in the JSON object, the corresponding field will remain null. This requires another post-deserialization type adapter.

final class SubstitutionTypeAdapterFactory
        implements TypeAdapterFactory {

    private final Function<? super Type, ? extends Type> substitute;

    private SubstitutionTypeAdapterFactory(final Function<? super Type, ? extends Type> substitute) {
        this.substitute = substitute;
    }

    static TypeAdapterFactory create(final Function<? super Type, ? extends Type> substitute) {
        return new SubstitutionTypeAdapterFactory(substitute);
    }

    @Override
    @Nullable
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        @Nullable
        final Type substitution = substitute.apply(typeToken.getType());
        if ( substitution == null ) {
            return null;
        }
        @SuppressWarnings("unchecked")
        final TypeAdapter<T> delegateTypeAdapter = (TypeAdapter<T>) gson.getDelegateAdapter(this, TypeToken.get(substitution));
        return delegateTypeAdapter;
    }

}

final class DefaultsTypeAdapterFactory
        implements TypeAdapterFactory {

    private final Function<? super Type, ? extends Type> substitute;
    private final LoadingCache<Class<?>, Collection<Map.Entry<Field, ?>>> fieldsCache;

    private DefaultsTypeAdapterFactory(final Function<? super Type, ? extends Type> substitute, final Function<? super Type, ?> toDefault) {
        this.substitute = substitute;
        fieldsCache = CacheBuilder.newBuilder()
                // TODO tweak the cache
                .build(new CacheLoader<Class<?>, Collection<Map.Entry<Field, ?>>>() {
                    @Override
                    public Collection<Map.Entry<Field, ?>> load(final Class<?> clazz) {
                        // TODO walk hieararchy
                        return Stream.of(clazz.getDeclaredFields())
                                .map(field -> {
                                    @Nullable
                                    final Object defaultValue = toDefault.apply(field.getGenericType());
                                    if ( defaultValue == null ) {
                                        return null;
                                    }
                                    field.setAccessible(true);
                                    return new AbstractMap.SimpleImmutableEntry<>(field, defaultValue);
                                })
                                .filter(Objects::nonNull)
                                .collect(Collectors.toList());
                    }
                });
    }

    static TypeAdapterFactory create(final Function<? super Type, ? extends Type> substitute, final Function<? super Type, ?> toDefault) {
        return new DefaultsTypeAdapterFactory(substitute, toDefault);
    }

    @Override
    @Nullable
    public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
        @Nullable
        final Type substitution = substitute.apply(typeToken.getType());
        if ( substitution == null ) {
            return null;
        }
        if ( !(substitution instanceof Class) ) {
            return null;
        }
        final Collection<Map.Entry<Field, ?>> fieldsToPatch = fieldsCache.getUnchecked((Class<?>) substitution);
        if ( fieldsToPatch.isEmpty() ) {
            return null;
        }
        final TypeAdapter<T> delegateTypeAdapter = gson.getDelegateAdapter(this, typeToken);
        return new TypeAdapter<T>() {
            @Override
            public void write(final JsonWriter out, final T value)
                    throws IOException {
                delegateTypeAdapter.write(out, value);
            }

            @Override
            public T read(final JsonReader in)
                    throws IOException {
                final T value = delegateTypeAdapter.read(in);
                for ( final Map.Entry<Field, ?> e : fieldsToPatch ) {
                    final Field field = e.getKey();
                    final Object defaultValue = e.getValue();
                    try {
                        if ( field.get(value) == null ) {
                            field.set(value, defaultValue);
                        }
                    } catch ( final IllegalAccessException ex ) {
                        throw new RuntimeException(ex);
                    }
                }
                return value;
            }
        };
    }

}

@AutoValue
abstract class AvoTestClass {

    abstract Optional<String> sval();

    static AvoTestClass create(final Optional<String> sval) {
        return new AutoValue_AvoTestClass(sval);
    }

    static Class<? extends AvoTestClass> type() {
        return AutoValue_AvoTestClass.class;
    }

}

public final class OptionalWithAutoValueTest {

    private static final Map<Type, Type> autoValueClasses = ImmutableMap.<Type, Type>builder()
            .put(AvoTestClass.class, AvoTestClass.type())
            .build();

    private static final Map<Class<?>, ?> defaultValues = ImmutableMap.<Class<?>, Object>builder()
            .put(Optional.class, Optional.empty())
            .build();

    private static final Gson gson = new GsonBuilder()
            .registerTypeAdapterFactory(new GsonJava8TypeAdapterFactory())
            .registerTypeAdapterFactory(SubstitutionTypeAdapterFactory.create(autoValueClasses::get))
            .registerTypeAdapterFactory(DefaultsTypeAdapterFactory.create(autoValueClasses::get, type -> {
                if ( type instanceof Class ) {
                    return defaultValues.get(type);
                }
                if ( type instanceof ParameterizedType ) {
                    return defaultValues.get(((ParameterizedType) type).getRawType());
                }
                return null;
            }))
            .create();

    @SuppressWarnings("unused")
    private static Stream<Optional<String>> test() {
        return Stream.of(
                Optional.of("ok"),
                Optional.empty()
        );
    }

    @ParameterizedTest
    @MethodSource
    public void test(final Optional<String> optional) {
        final AvoTestClass before = AvoTestClass.create(optional);
        final String json = gson.toJson(before, AvoTestClass.class);
        final AvoTestClass after = gson.fromJson(json, AvoTestClass.class);
        Assert.assertEquals(before, after);
    }

}

This solution is reflection-based heavily, but it's just a work-around if the generators cannot do the job (again, not sure if they can be configured so that there are no such issues).