Picker API recent issues when using existing access token

Question

Our app has a flow in which a user integrates his google account with the following scopes:

• email
• profile
• https://www.googleapis.com/auth/drive.readonly
• https://www.googleapis.com/auth/drive.file
• https://www.googleapis.com/auth/drive.metadata

After doing this integration we store the token and refresh token data on our database. Later, we reuse that information to open the google drive picker without asking the user to be logged in.

We refresh the token in the backend if the token is expired. After the token is updated, we use it in the frontend like this:

let picker = new google.picker.PickerBuilder()
        .enableFeature(google.picker.Feature.SUPPORT_DRIVES)
        .addView(google.picker.ViewId.DOCUMENTS)
        .setAppId(app_id)
        .setOAuthToken(auth.google_access_token)
        .setDeveloperKey(auth.google_file_picker_key)
        .setCallback(this.pickerCallback(resolve, reject))
        .build();

picker.setVisible(true);

So, this USED to work, but suddenly stopped.

We received one email from google 15 days ago about changes in scopes, that since we use drive.readonly, that our app will become unverified. Also saw this recently: https://www.googlecloudcommunity.com/gc/Workspace-Q-A/Drive-API-Auth-Scope-Changes-Handling-Changing-Remvoing-Scopes/m-p/674909

While the backend part of the integrations seems to be working properly, for some reason the picker part is not accepting the provided oauth token.

If it matters, when first authorizing we use the 'offline' access type and prompt: 'consent select_account'.

Now, when I try to open the file picker I'm presented with a login screen if I'm not logged in, a 403 error if I login with a different user (403. That’s an error. We're sorry, but you do not have access to this page. That’s all we know.) and it works if I login with the same user I originally authorized.

Does anyone know if something changed recently on google side?