Connection Interupted in for Loop Making Post Request to Infura

15 Views Asked by At

I made a script and I am doing a few things in it. I am uploading 10,000 documents and a JSON files to IPFS through Infura, and I am writing data to an Excel speadsheet. The Excel part works fine, I am only mentioning that in case it matters. I start the loop, and pretty early on I get raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer')) I have seen similar posts, including this, but I read somewhere else that if you strategically put sleep(0.01) around your code, it will help with this. While I have seen improvements, this issue still persists. I am not sure if I placed them right or if I can do something else. This is the short version of my script:

import time
import requests
from xlwt import Workbook
import json

### IPFS Integration
for i in range(1, 10000 + 1):
    # Create image file object
    image_file = {
        'file': open('images/' + str(i) + '.png', 'rb')
    }
time.sleep(0.01)

### Upload image to IPFS
image_response = requests.post(endpoint + '/api/v0/add', files=image_file, auth=(project_id, project_secret))

time.sleep(0.01)

### Change metadata image url
# Create raw JSON file
json_file = open('newjson/' + str(i) + '.json', 'r+')
raw_json = json.load(json_file)
json_file.close()

# Change image URL
raw_json['image'] = image_url

time.sleep(0.01)

# Save raw JSON
json_file = open('newjson/' + str(i) + '.json', 'w')
json.dump(raw_json, json_file)
json_file.close()

# Create metadata file object
meta_file = {
    'file': open('newjson/' + str(i) + '.json', 'rb')
}

time.sleep(0.01)

### Upload metadata to IPFS
meta_response = requests.post(endpoint + '/api/v0/add', files=meta_file, auth=(project_id, project_secret))

params = {
    'arg': meta_cid
}

time.sleep(0.01)

meta_info_response = requests.post(endpoint + '/api/v0/cat', params=params, auth=(project_id, project_secret))
meta_sheet.write(i, json_col, meta_info_response.text)

I got rid of a lot of variables for privacy purposes, but how does this look? I copied and pasted a lot so I may have forgotten some chunks. But am I putting the sleeps in the right spot? Usually I will get an error from round 9 till 25. Please let me know if I can get some help, thank you!

0

There are 0 best solutions below