How can we write code using fsspec to read data from gcp on our local system

109 Views Asked by At

I tried writing below code but facing below errors.

credentials = service_account.Credentials.from_service_account_file('data.json')

fs = fsspec.filesystem('gcs', gcs=client, token=credentials)

csv_files = fs.glob("bucket-name/*.csv")

dfs = []
for file in csv_files:
    with fs.open(file) as f:
        df = pd.read_csv(f)
        dfs.append(df)

df = pd.concat(dfs, ignore_index=True)

print(df.head())

But when i tried writing this code...it is able to return me all bucket name on my gcp a/c but it is not able to read the different types of data file present inside this and it gives me an error of SSL.

RefreshError: ('invalid_scope: Invalid OAuth scope or ID token audience provided.', {'error': 'invalid_scope', 'error_description': 'Invalid OAuth scope or ID token audience provided.'})

Is it because i did not install google cloud cli and directly trying to use it with help of key.json.

Also when i try to install gsspec library as well it will give us some modules error. Please note: I am using 3.12 version of python in order to solve above question.

0

There are 0 best solutions below