I believe this problem is pretty common and struggle with it for a number of years. Quite surprisingly, nothing has been done about it by Google so far.
My game supports playing over Google Play Service (RealTimeMultiplayer), but it also can play locally (no network), or over Bluetooth, where sign in to Google is not required.
When a player receives a notification, accepts the invitation and the application starts, it needs an indication (an extra field in Intent maybe) that the application was activated by invitation (it will be great to provide GoogleSignInAccount and maybe also Invitation, given than both are Parcelable). Having such an indication, the application would sign the player to Google, accept the invitation and go straight to game, otherwise it will just show the startup screen without requesting to sign in. Unfortunately, I can't see such an indication.
I tried getLastSignedInAccount and silentSignIn in hope that will bring up the account related to the invitation. Nope! Typically, in case of activated by invitation getLastSignedInAccount returns null, while silentSignIn fails (the code is obviously 4: SIGN_IN_REQUIRED).
Yet another desperate attempt. When application starts by launcher, it uses category android.intent.category.LAUNCHER, otherwise android.intent.category.INFO. However it is not 100% proof, as when application starts from Google Play Store app, android.intent.category.INFO is also used.