Why is there a solid black rectangle obscuring the rightmost 60% of my app's screen on a Google Pixel 5?

Question

Does anybody know why a Google Pixel 5 and a Google Pixel 5e (API 30) might have a solid black overlay completely obscuring the rightmost 60% of the screen when viewing the otherwise normal in-app screenshots produced by Automatic Robo Script in Firebase Test Lab?

We have had no such display issues testing this app, which was built in Unity 2017 and exported to Android Studio, before uploading to Firebase Test Lab as an .aab build on 20 other devices, several of which appear correctly in Firebase Test Lab including other Pixels.

Neither have we had display issues when specifically testing API 30 in Android Studio Pixel 2, Pixel 4, and Pixel XL emulators.

Not sure if it's a Pixel 5 feature that the robot gets stuck on and a real user would know to swipe away the screen or something. Any help short of buying a Pixel 5 to debug on would be massively appreciated.

Image produced by firebase robo script exploring the app

-------------- Update: ------------------

I've been using my 5 daily Robo tests on Pixel 5 to try to debug this and the first thing I thought I'd address were some build warnings that were showing in the Play Console's Pre-Launch Report (but not in Android Studio or in Firebase's Test Results) of type "Unsupported API":

StrictMode policy violation: android.os.strictmode.NonSdkApiUsedViolation: Landroid/view/textclassifier/logging/SmartSelectionEventTracker$SelectionEvent;->selectionAction(III)Landroid/view/textclassifier/logging/SmartSelectionEventTracker$SelectionEvent;
    at android.os.StrictMode.lambda$static$1(StrictMode.java:428)
    at android.os.-$$Lambda$StrictMode$lu9ekkHJ2HMz0jd3F8K8MnhenxQ.accept(Unknown Source:2)
    at java.lang.Class.getDeclaredMethodInternal(Native Method)
    at java.lang.Class.getPublicMethodRecursive(Class.java:2075)
    at java.lang.Class.getMethod(Class.java:2063)
    at java.lang.Class.getMethod(Class.java:1690)
    at cEa.a(PG:9)
    at cEi.a(PG:12)
    at org.chromium.content.browser.selection.SmartSelectionClient.<init>(PG:5)
    at cEQ.a(Unknown Source:7)
    at org.chromium.android_webview.AwContents.e(PG:245)
    at org.chromium.android_webview.AwContents.d(PG:210)
    at org.chromium.android_webview.AwContents.<init>(PG:77)
    at to.run(PG:15)
    at aJK.a(PG:13)
    at aJL.run(Unknown Source:2)
    at org.chromium.base.task.PostTask.b(PG:16)
    at aJK.a(PG:7)
    at com.android.webview.chromium.WebViewChromiumFactoryProvider.b(PG:6)
    at com.android.webview.chromium.WebViewChromium.init(PG:105)
    at android.webkit.WebView.<init>(WebView.java:422)
    at android.webkit.WebView.<init>(WebView.java:348)
    at android.webkit.WebView.<init>(WebView.java:331)
    at android.webkit.WebView.<init>(WebView.java:318)
    at android.webkit.WebView.<init>(WebView.java:308)
    at com.google.android.gms.ads.internal.webview.ac.<init>(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:1)
    at com.google.android.gms.ads.internal.webview.s.a(Unknown Source:0)
    at com.google.android.gms.ads.internal.util.bi.b(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:3)
    at com.google.android.gms.ads.internal.webview.v.a(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:2)
    at com.google.android.gms.ads.internal.js.l.<init>(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:4)
    at com.google.android.gms.ads.internal.js.n.run(Unknown Source:0)
    at android.os.Handler.handleCallback(Handler.java:873)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at qd.a(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:0)
    at com.google.android.gms.ads.internal.util.f.a(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:1)
    at qd.dispatchMessage(:com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:0)
    at android.os.Looper.loop(Looper.java:193)
    at android.app.ActivityThread.main(ActivityThread.java:6718)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

The one SDK we're using is Google Mobile Ads (Version 5.4.0) for Unity which has its own build.gradle file outside our project module in Android Studio, the default settings of which (when exported via Unity 2017 which does not support API 30) are as follows:

    buildToolsVersion '29.0.2'
    targetSdkVersion 19

In the instance of the black screen state I had updated these as per Android Studio's recommendation (where targetSdkVersion 19 showed red) to:

    buildToolsVersion '31.0.0-rc3'
    targetSdkVersion 30

Upon setting these back to the apparently corrupt Unity defaults and re-uploading to Robo Test it worked! The Pixel 5 showed the full screen. I have a feeling something else is going on however as while reversing the process brought back the black screen, pushing it forward a second time did not resolve it. I'm now out of my daily tests.

Answer 1

The software Google uses in it's Pre-Launch Report to aid developers in highlighting device specific UI issues is itself unstable, and contrary to the recommendations within the console, should not be used to do so with Unity3D projects, despite the fact that Unity projects constitute the majority of games uploaded to Google Play.

Thanks to the kind, and charitable help of the guys at Mindful QA we have been able to test this on a number of physical devices, including a Pixel 5 and can confirm that the black rectangle issue is exclusive to Google's Firebase Test Lab which is the engine behind the Pre-Launch-Report screens in Google Play, it's physical devices that run API 30, and their inability to draw coherent screens across all Unity3D projects.

We have reproduced this consistently even with completely empty Unity3D projects. We have also reproduced this with Unity3D projects built in both Unity 2017 before being exported to Android Studio, and Unity3D projects built entirely in the latest version of Unity (2021.1.6f1).

I had done my best to report this to the Firebase Team through the recommended process of their Google Groups page but after 3 weeks I did not receive a response. Google Play Developer Support has now investigated the issue, but is not able to confirm the bug, nor log it with the appropriate team to be fixed so I imagine it will persist for a while. Until then, hopefully this can help developers that lose weeks of their own time trying to understand why there is a black rectangle in their apps uploaded to Google Play, when they follow the recommended process for highlighting UI issues in the Play Console.

The steps for reproducing this with an Empty Unity3D project are as follows:

1. Download Unity 2021.1.6f1 and make a new folder for the project on your hard drive.
2. Within Unity Hub ensure Unity 2021 has Android Build support and modules added for the SDK, NDK and JDK: Unity Hub Settings
3. Inside Unity -> Edit -> Preferences, ensure the External Tools for the SDK, JDK, and NDK are all set to "Install With Unity (recommended)" Unity external Path config
4. In Unity File -> Build Settings -> Player Settings -> Other Settings ensure the Target SDK is set to Automatic (Highest Installed) and that Android 11 appears within the list: Player Settings
5. Build the project as an .apk.
6. Go to https://console.firebase.google.com and select an existing Project or create a new Project.
7. Select Test Lab from the left panel and choose Run a Test, before proceeding to upload the .apk.
8. Select Customize and use the filter to find any of the physical devices running API 30, at the time of writing this could be a Pixel 5e or a Pixel 4 set to use API 30. The orientation can be either Portrait or Landscape.
9. Choose Confirm and then Start 1 Test. This takes about 5 minutes.
10. When the test is finished select the test and then press ScreenShots and observe the black rectangle Empty Project Black Rectangle

This never happens on any of the devices running API 29 or lower, in total the devices we have tested on include:

Firebase Success (no black rectangle): LG H932 (API 26) Nexus 5 (API 23) MHA-L29 (API 24) H8216 (API 28) Pixel 2 (API 27) Pixel (API 26) ONEPLUS A5010 (API 28) SM-G955F (API 26) Galaxy S9 (API 26) Pixel 3 (API 28) Redmi 6A (API 27) Nokia 1 (API 27)

Firebase Fail (black rectangle): Pixel 4 (API 30) Pixel 5e (API 30) Pixel 5 (API 30)

Where the API 30 devices fail consistently on about 19/20 tests. Obviously with 5 free tests per day this data takes a long time for a developer to collect, so I hope it can be helpful to somebody trying to either fix this bug or understand why their own project has black rectangles when they upload it to Google Play.