Is there a way to run multiple suites in parallel using the select-suite option of saucectl for playwright tests

59 Views Asked by At

I am trying to run multiple test suites - each suite on a separate browser, that are defined in my config file using the npx saucectl run --select-suite "SuiteName" command from inside a Makefile.

My .sauce/config.yml is structured this way:

apiVersion: v1alpha
kind: playwright
sauce:
  region: us-west-1
  tunnel:
    name: 'playwright-tunnel'
    owner: '$SAUCE_USERNAME'
  concurrency: 10
  metadata:
    tags:
      - regression
  launchOrder: fail rate
playwright:
  version: package.json
  configFile: playwright.config.js
rootDir: ./
env:
  AWS_SDK_LOAD_CONFIG: 1
  AWS_SHARED_CREDENTIALS_FILE: ".aws-session.saucelabs.ini"
  AWS_CONFIG_FILE: ".aws-config.saucelabs.ini"
npm:
  dependencies:
    - date-fns
    - date-fns-tz
    - events
suites:
  - name: "Login Tests"
    platformName: "Windows 11"
    screenResolution: "1920x1080"
    testMatch: ['tests/login/.*.js']
    params:
      browserName: "chromium"
      project: "chromium"
  - name: "Login Tests Firefox"
    platformName: "Windows 11"
    screenResolution: "1920x1080"
    testMatch: ['tests/login/.*.js']
    params:
      browserName: "firefox"
      project: "firefox"

My Makefile looks like this:

procs = $(shell ps -ef | grep 'bin/sc' | grep -v grep | awk '{ print $$2 ; }')
killcmd = $(if $(procs), "kill" "-9" $(procs), "echo" "no matching processes")
SUITE_NAME ?= "Login"

install:
  rm -rf node_modules || true
  yarn
  npm install saucectl@latest --no-save

start-tunnel:
    sc-tunnel/osx/bin/sc -c sc-tunnel/osx/tunnel-config.yml

stop-tunnel:
    @echo 'sauce tunnel processId: ['$(procs)'] stopped'
    @$(killcmd)

run-test :
  npx saucectl run --ccy 4 --select-suite "${SUITE_NAME}"

I am passing the SUITE_NAME from my Jenkinsfile, and am able to read it successfully. The --select-suite only accepts a string.

I have tried out a few options like passing the SUITE_NAME as a regex or by trying to call 2 npx commands inside the make run-test, but each time either just the string matched suite is run or all the test suites get executed.

I have read through the documentation provided on saucelabs for playwright but looks like there is no direct way of doing this.

I would like to hear from others and try out any possible suggestion on this?

0

There are 0 best solutions below