Accessibility Testing using Pa11y 5 on a page with Popup / Overlay

Question

I have a page that I'm able to get accessibility test report from Pa11y 5 using actions.

When I click on a button on that specific page, I get a popup/overlay, I would like Pa11y to sniff that popup/overlay page and report on accessibility metrics, but currently Pa11y 5 is only able to provide me for the main parent page ignoring any reports on the html in the popup page? Is there a way to achieve this, tell Pa11y to switch to popup and sniff on that popup html and report that.

Popup/overlay contains div[role='dialog'] as it is a modal dialog made out of aria.

I'm using the latest Pa11y, hence I keep mentioning it as Pa11y 5. I have not used Pa11y4, so cannot comment if this works with Pa11y 4.

Any help/advise is sincerely appreciated.

Update: As requested, below is my complete (relevant) part of the code

const PageOptions1 = {  
    timeout: 30000,
    userAgent: 'A11Y TESTS',
    actions: [
        'screen capture screenshots/001-DefaultView.png'                    
    ]
};
const PageOptions2 = {
    timeout: 35000, 
    userAgent: 'A11Y TESTS',
    rootElement: 'div[role="dialog"]',
    actions: [
        'click element button[data-automation-id="ccbutton"]',
        'wait for element div[role="dialog"] to be added',
        'screen capture screenshots/002-Popup.png',
        'click element i.fa-close',
        'screen capture screenshots/002-DefaultView.png'
    ]
};

async function runPa11y(navigateUrl) {
    try {
        const results = await Promise.all([
            pa11y(navigateUrl, PageOptions1),
            pa11y(navigateUrl, PageOptions2),           
        ]);

        LogResults(results);
    } catch (error) {
        console.error("Error: " + error.message);
    }
}

runPa11y("Url to navigate");

Answer 1

Thanks for your updates! You're doing the right things and everything looks like it's working as expected.

Pa11y will perform all of the Actions before running your test, so it's opening the modal dialog, immediately closing it again, and then running the test without it.

Break PageOptions2 into smaller units so that your last Action is the state you want to test against, and everything should be ok.