“ERROR 429: Too Many Requests.” occurs when using wget to download big file (15G) from Zenodo

Question

I want download the LoDoPaB-CT dataset from Zenodo from here.

The 'test' and 'validation' data have been successfully downloaded. But when downloading the train data, which are very large. I always get the error ERROR 429: Too Many Requests after starting download sometime. That's to say when the download size is about 1G~3G, the 429 error occurrs. What should I do to solve this? Thanks!

Answer 1

Try to use "--wait" and "--continue" options of wget. For example:

wget --wait 10 --random-wait --continue ${your_url}

Will wait a random of 5 to 15 seconds between retrievals, and resume getting a partially-downloaded file.