I have a Python script that collects source links and puts each link as a new line in a text file. This file is updated continuously as I generally run the Python script for most of the day. The textfile contains duplicate links, so that I usually have to manually remove duplicates before downloading.
Right now I manually put the textfile in Jdownloader and have the downloads saved to the folder on my PC, skipping any duplicates (downloads from before). However, this is a manual process, and by the time I get to it, some of the links are dead.
Is there a way to automate this? Like every few hours it can use the textfile at that time, download all the links to the folder (no subfolders), and skip all previously downloaded content (same filename)? Or is Jdownloader manually my best option?
I think you need to use a cron job. You can do so with Windows' Task Scheduler. With it you can schedule your script execution.
This tutorial may help you.
In your script you can download each file using the Requests library