Does Avro Schema Allow Conditional Fields?

438 Views Asked by At

I'm planning to use Avro's Java API to build out a schema (preferably by using a tool which converts POJOs to AVSC).

  1. Does Avro schema allow conditional fields like JSON schema does using allOf, anyOf, dependencies etc.?
  2. If not, what's the best way to achieve conditional validation in Avro?
  3. Alternatively, is there a tool to convert a JSON schema to AVSC (I already have a JSON schema defined with the conditional validation I need)?

For example:

  • If field1 is present, then field2 should be present too.
  • field3 should always be present.
  • If field2 is present, then field4 which is an enum of [a,b,c] can only have the value [a,b].
1

There are 1 best solutions below

0
On

You can represent optional fields in Avro by making the field type a union between the actual type and null, e.g.:

{
    "name": "username",
    "type": [
      "null",
      "string"
    ],
    "default": null
},

1. Does Avro schema allow conditional fields like JSON schema does using allOf, anyOf, dependencies etc.?

There is no way to make the schema for one field dependent on the value of another field.

2. If not, what's the best way to achieve conditional validation in Avro?

Simply make the schema as retrictive as possible and then for any rules that you can't capture in the schema, simply apply them as part of a post-construction validation step.

E.g. taking your examples:

  • If field1 is present, then field2 should be present too.

Simply make field1 and field2 optional, and apply the check on field2 during validation.

  • If field2 is present, then field4 which is an enum of [a,b,c] can only have the value [a,b].

Again, simply apply the check on field4 during validation.

3. Alternatively, is there a tool to convert a JSON schema to AVSC (I already have a JSON schema defined with the conditional validation I need)?

I doubt it. In any case, even if one did exist, given that Avro Schema is less powerful than JSON Schema, it woudn't solve your problem.