How to ignore FAIL during conftest policy check in atlantis

Question

I am still in the testing phase and would like to allow merging of my PR even if the policy_check is FAIlING. What flag I can pass to enable that ? My current config:

  repos:
  - id: /.*/
    workflow: custom
    apply_requirements: [mergeable]
    allow_custom_workflows: true
    allowed_overrides: [apply_requirements, workflow]
  policies:
    owners:
      users:
        - XXX
    policy_sets:
      - name: conftest
        path: /atlantis/conftest_policies/policies/
        source: local
  workflows:
    custom:
      plan:
        steps:
          - init
          - plan
      policy_check:
        steps:
          - show
          - policy_check:
              extra_args: ["--update", "git::https://${serviceVariable.SAMPLE_TOKEN}@github.com/Company/conftest-policy.git", "--all-namespaces" ]

This is my output during atlantis plan:

exit status 1
Checking plan against the following policies: 
  conftest
? - <redacted plan file> - gcp.common - no policies found
FAIL - <redacted plan file> - gcp.iam - Service Account User and Service Account Token Creator roles are prohibited at the project level and must be assigned to specific service accounts. service_account_iam_roles uses the role roles/iam.serviceAccountUser.

38 tests, 36 passed, 1 warning, 1 failure, 0 exceptions`

Answer 1

I was able to solve it by passing '--no-fail' flag

Answer 2

Based on the documentation

Any failures need to either be addressed in a successive commit, or approved by a blessed owner. This approval is independent of the approval apply requirement which can coexist in the policy checking workflow. After an approval, the apply can proceed.

screenshot from Atlantis docs

Answer 3

Another method is to use warn rule instead of deny rule in the rego policy

See this example where the terraform_remote_state is found and warned on instead of denied.

package evil_sources

import future.keywords

data_sources := {"terraform_remote_state"}

warn contains sprintf(message, [key]) if {
    message := "This data source is evil (%s)"

    input.data[key]
    key in data_sources
}

This way you can simply throw a warning (exit 0) to avoid having to set --no-fail globally on conftest