XCUITest needs to be ran twice to be successful because of app reboot

Question

I have a VoIP application that somewhere during the on boarding process resets itself. Because of this the first time I run the UI test it will always fail and the second time it will always pass.

I use this UI Test to create screenshots with snapshot (fastlane) and of course I can tell snapshot to try at least twice. But I also want to run this UI test whenever somebody merges to develop so I know it's broken, instead of fixing the scripts manually when we want to release. So I really would like some kind of way to let this test pass from a clean Simulator.

Is there any way possible?

Answer 1

I finally figured out a way to more or less get around this problem. The problem had to with the state of the application. You cannot influence the application directly from the UI test but you can set some variables.

First in my unit test

XCUIApplication *app = [[XCUIApplication alloc] init];
[Snapshot setupSnapshot:app];
app.launchEnvironment = @{ SkipOnboarding: @YES };
[app launch];

Then in AppDelegate:

NSDictionary *environment = NSProcessInfo.processInfo.environment;
if (environment[SkipOnboarding]) {
    [SnapShotHelper skipOnboardingAndLogin];
}

SkipOnboarding is a const I created myself. It's really just a string.

The snapshot helper sets some defaults needed to app the boot like the user was already logged in. Of course there are some caveats:

• You just lost the option to snapshot the non-logged in state
• Code inside the application just for running tests

But since we're not snapshotting the login procedure it's fine. Alternatively we could create two separate UI tests if that was needed.