ceberus: How to ignore a field based on yaml comment?

Question

Overview

I have a lot of .yaml files, and a schema to validate them. Sometimes, a "incorrect" value, is in fact correct.

I need some way to ignore some fields. No validations should be performed on these fields.

Example

  ## file -- a.yaml
  some_dict:
      some_key: some_valid_value

  ## file -- b.yaml
  some_dict:
      some_key: some_INVALID_value # cerberus: ignore

How can I do this?

Answer 1

Quick Answer (TL;DR)

• The "composite validation" approach allows for conditional (context-aware) validation rules.
• The python cerberus package supports composite validation "out of the box".
• YAML comments cannot be used for composite validation, however YAML fields can.

Detailed Answer

Context

• python 2.7
• cerberus validation package

Problem

• Developer PabloPajamasCreator wishes to apply conditional validation rules.
• The conditional validation rules become activated based on the presence or value other fields in the dataset.
• The conditional validation rules need to be sufficiently flexible to change "on-the-fly" based on any arbitrary states or relationships in the source data.

Solution

• This approach can be accomplished with composite data validation.
• Under this use-case, composite validation simply means creating a sequential list of validation rules, such that:
  • Each individual rule operates on a composite data variable
  • Each individual rule specifies a "triggering condition" for when the rule applies
  • Each individual rule produces one of three mutually-exclusive validation outcomes: validation-success, validation-fail, or validation-skipped

Example

Sample validation rules

- rule_caption:     check-required-fields
  rule_vpath:       "@"
  validation_schema:
    person_fname:
      type: string
      required: true
    person_lname:
      type: string
      required: true
    person_age:
      type: string
      required: true

- rule_caption:     check-age-range
  rule_vpath:       '@|@.person_age'
  validation_schema:
    person_age:
      "min": 2
      "max": 120

- rule_caption:     check-underage-minor
  rule_vpath:       '[@]|[? @.person_age < `18`]'
  validation_schema:
    prize_category:
      type: string
      allowed: ['pets','toys','candy']
    prize_email:
      type:     string
      regex:    '[\w]+@.*'

• The code above is a YAML formatted representation of multiple validation rules.

Rationale

• This approach can be extended to any arbitrary level of complexity.
• This approach is easily comprehensible by humans (although the jmespath syntax can be a challenge)
• Any arbitrarily complex set of conditions and constraints can be established using this approach.

Pitfalls

• The above example uses jmespath syntax to specify rule_vpath, which tells the system when to trigger specific rules, this adds a dependency on jmespath.

See also

• complete code example on github