I have a suite of 1600 Playwright tests that run perfectly well on the Chromium browser, both locally and in an Azure release pipepine.
For some reason, a large volume of random tests will fail when run locally. They will time out when simply trying to find the username or password field (some very shared code!) but will pass when run again.
When run in the pipeline, a large number of tests will fail for the same reason. The test run will then eventually freeze and we are then forced to manually stop the run (this could be after hours of being in a frozen state, days if run over the weekend).
Is there something special that I need to factor in when running these tests on Firefox? I have been operating under the assumption that the tests should just "work", but is this not correct?
Thanks!
EDIT: It appears that the tests will run successfully in Firefox, when not run in parallel. Is there something extra I need to factor in when running Firefox tests in parallel?
Unfortunately no. Otherwise, cross-browser testing problems wouldn't exist.
Chrome and FF use different JS engines and may render UI differently and devs may add browser-specific behaviors for each browser for JS/CSS/etc.
So the answer is - you have to debug all Firefox tests locally.
P.SP Also, next time you come to Safari (or WebKit for Playwright), you'll see that the same test is perfectly fine on MacOS but fails in Docker on Linux/Windows - another level of fun