How to configure pa11y timeout under actions for a json config file to override default 30 seconds timeout

Question

We have a jenkins scripted pipeline to run pa11y automated testing against a set of authenticated URLs using Actions. We run it on linux and the config file is a json format file. Below is the file :-

{
    "chromeLaunchConfig": {
        "args": [
            "--no-sandbox",
            "--disable-setuid-sandbox",
            "--disable-dev-shm-usage"
        ]
    },  
"actions":[     
                "set field #loginUserId to ***",
                "set field #loginPassword to ***",
                "click element #loginButton_0 to ***",  
                "wait for url to be https://www.qa.***.com/account/dashboard"  
          ]
}

Here is the actual command line script that invokes this config json file to run the tests :-

pa11y --config config.json --threshold 500 -t 120000 --include-notices --include-warnings --reporter cli https://www.qa.***.com/login > results.json

Most of the times it works just fine but few times it just times out with below error :-

"Error: TimeoutError: waiting for function failed: timeout 30000ms exceeded"

I understand that default timeout is 30 seconds and hence i am overriding that using -t in command line with 120 seconds. But looks like under actions where its "wait for url to be *****", it still picks up default 30 seconds and fails sometimes with timeout. How can i configure timeout option for actions in a json file to avoid this error?

Answer 1

Pa11y has a timeout flag/config option that allows you to specify the time in milliseconds that pa11y will wait for a test to finish before giving up.

This timeout value is for the entire test run, so that will include the time to initialise Chrome through the puppeteer library, load the page, carry any actions that have been specified, and finally run the automated accessibility test. As of pa11y 6.1.0 the default timeout is 60 seconds.

There's an additional timeout for each individual action that the browser takes, this is set by the puppeteer library and it's set to 30 seconds. Although this value can be changed in puppeteer, I don't think that this timeout is exposed in pa11y so it's not something that can be configured directly.

Some alternatives could be:

• Make the site faster so the wait for action takes less than 30 seconds
• Pa11y by default will create a new puppeteer instance, but you can pass it an existing puppeteer instance with a page already loaded. You could use write a short node script that starts puppeteer, executes the desired actions using puppeteer commands (with a timeout higher than 30 seconds for each of them), and then finally calls pa11y: https://github.com/pa11y/pa11y/tree/9cf10d4d511ccb0e5129495968a24b6512bfe6ee#browser-browser-and-page-page