AWS Device Farm XCTest UI generated .xctestrun file is not valid

Question

Device Farm generates a .xctestrun file and the location is stored at $DEVICEFARM_XCUITESTRUN_FILE. However, my test fails everytime

xcodebuild: error: Failed to build workspace temporary with scheme Transient Testing.
    Reason: There are no test bundles available to test.
    Recovery suggestion: Ensure that all package dependencies have resolved and that there are no missing test bundles in the active scheme or test plan.

I print out the .xctestrun file in the yaml file and see that it's contents are not in the same format as the .xctestrun file created by Xcode during my build. Specifically the top-level object of the generated file shows up as a child in the Xcode file with the path TestConfigurations/TestTargets. The generated file is missing lots of top level objects like TestConfigurations, ContainerInfo, TestPlan, and CodeCoverageBuildableInfos.

Has anyone had success running iOS UI Tests in a custom environment using the generated .xctestrun file? According to the man xcodebuild.xctestrun output, the generated file is not in the correct format.

I've tried downloading a copy of the xctestrun file and using it locally and the tests also fail. I am able to successfully run the test locally when using the .xctestrun file created by Xcode.

Answer 1

In order to make this work I had to bypass all the automatic stuff that Device Farm does. I upload my own .xctestrun file instead of using the one generated by Device Farm. Then in my YAML test spec, I create a folder /tmp/root and copy my .xctestrun file into it. Then I create /tmp/root/Debug-iphoneos and copy both my app and test package .app files into that folder. This mimics my local environment exactly, and it successfully runs the test