OWASP ZAP baseline scan returns unexpected error 1 in CI/CD pipeline

Question

I am using a docker image from OWASP in my pipeline to scan my web app and produce a HTML report, and I am encountering a problem I've spent the whole day trying to solve.

When running the scan job, it will successfully scan the website, but immediatly after executing the scan command, the job will stop and return "error: job failed: exit code 1".

This happens without precisions on what has failed in the command.

Here is the code of my job :

zap_scan:
  stage: owasp
  image:
    name: owasp/zap2docker-stable
  script:
    - mkdir /zap/wrk
    - /zap/zap-baseline.py -t http://webURL.com -g gen.conf -r /zap/wrk/report.html
  artifacts:
    paths:
      - /zap/wrk/report.html

Normally, this shouldn't be returning an error as a I have tested the scan command by running this on a locally built, identic docker image and I have encountered no issues (ie the scan and the file were generated properly).

Here is the open source code of the zap-baseline.py script

By looking into this, I've found that the script can return error 1 if fail_count is different than 0.

I do not understand why the script behave differently on a local docker image and in a pipeline, can you help me please ?

Answer 1

Gitlab sends an automatic error when a script returns 1, which was the case because the scan had warnings. Adding the option -I fixed the error.

Answer 2

You will need to share the full output from the scan (obfuscating anything sensitive) in order for us to be able to help more.

btw, why are you specifying -g gen.conf ? That generates a config file, which you do not appear to be using..

Answer 3

Below is the working gitlab job for zap scan.

zap-scan:
  stage: owasp
  image:
    name: owasp/zap2docker-stable:latest
  before_script:
    - mkdir -p /zap/wrk
  script:
    - echo "Starting zap-api-scan.py baseline scan"
    - cp ./owasp-zap/warn-levels.config /zap/wrk
    - zap-api-scan.py -t http://webURL.com -c warn-levels.config -w testreport.md -r testreport.html -S -I
    - cp /zap/wrk/testreport.md testreport.md
    - cp /zap/wrk/testreport.html testreport.html
    - grep -Po '\| High \| 0 \|' testreport.md
    - grep -Po '\| Medium \| 0 \|' testreport.md
  artifacts:
    when: always
    paths:
      - zap.out
      - testreport.md
      - testreport.html

Note the flag -c with warn-levels.config this extra config you can provide to configure the errors which you want to ignore.

In our case, we wanted to exclude

WARN-NEW: Spring Actuator Information Leak [40042] x 1

So, the file warn-levels.config looks like below

40042   IGNORE  (Spring Actuator Information Leak - Active/release)

Reference for the various flags https://www.zaproxy.org/docs/docker/api-scan/#usage