I am trying to run multiple test suites - each suite on a separate browser, that are defined in my config file using the npx saucectl run --select-suite "SuiteName" command from inside a Makefile.
My .sauce/config.yml is structured this way:
apiVersion: v1alpha
kind: playwright
sauce:
region: us-west-1
tunnel:
name: 'playwright-tunnel'
owner: '$SAUCE_USERNAME'
concurrency: 10
metadata:
tags:
- regression
launchOrder: fail rate
playwright:
version: package.json
configFile: playwright.config.js
rootDir: ./
env:
AWS_SDK_LOAD_CONFIG: 1
AWS_SHARED_CREDENTIALS_FILE: ".aws-session.saucelabs.ini"
AWS_CONFIG_FILE: ".aws-config.saucelabs.ini"
npm:
dependencies:
- date-fns
- date-fns-tz
- events
suites:
- name: "Login Tests"
platformName: "Windows 11"
screenResolution: "1920x1080"
testMatch: ['tests/login/.*.js']
params:
browserName: "chromium"
project: "chromium"
- name: "Login Tests Firefox"
platformName: "Windows 11"
screenResolution: "1920x1080"
testMatch: ['tests/login/.*.js']
params:
browserName: "firefox"
project: "firefox"
My Makefile looks like this:
procs = $(shell ps -ef | grep 'bin/sc' | grep -v grep | awk '{ print $$2 ; }')
killcmd = $(if $(procs), "kill" "-9" $(procs), "echo" "no matching processes")
SUITE_NAME ?= "Login"
install:
rm -rf node_modules || true
yarn
npm install saucectl@latest --no-save
start-tunnel:
sc-tunnel/osx/bin/sc -c sc-tunnel/osx/tunnel-config.yml
stop-tunnel:
@echo 'sauce tunnel processId: ['$(procs)'] stopped'
@$(killcmd)
run-test :
npx saucectl run --ccy 4 --select-suite "${SUITE_NAME}"
I am passing the SUITE_NAME from my Jenkinsfile, and am able to read it successfully. The --select-suite only accepts a string.
I have tried out a few options like passing the SUITE_NAME as a regex or by trying to call 2 npx commands inside the make run-test, but each time either just the string matched suite is run or all the test suites get executed.
I have read through the documentation provided on saucelabs for playwright but looks like there is no direct way of doing this.
I would like to hear from others and try out any possible suggestion on this?