Unable to read csv from S3 presigned url directly in jupyter notebook

Question

Hi is there anyway to open csv file from s3 presigned url in a script rather than downloading it from browser! I recieve a presigned s3 url every hour on gmail. I have to download it manually everytime. I decided to automate the process by scraping my emails, have reached the step where I am able to get the fresh presigned link but unable to open the csv file, rather it returns a script. Please help.

Answer 1

You can use requests to perform a GET request.

From the AWS page explaining boto3 and presigned URLs:

The user can download the S3 object by entering the presigned URL in a browser. A program or HTML page can download the S3 object by using the presigned URL as part of an HTTP GET request.

The following code demonstrates using the Python requests package to perform a GET request.

import requests    # To install: pip install requests

url = create_presigned_url('BUCKET_NAME', 'OBJECT_NAME')
if url is not None:
    response = requests.get(url)

Answer 2

You can read it directly using pandas read_csv. From the docs:

So you can simply:

import pandas as pd
df = pd.read_csv('https://...')