bash: ERROR: batch file could not be read

Question

Inside C:\cygwin64\home\Administrator I have 3 files

script.sh
links_da_scaricare.txt
cookies.txt

To download from dplay I use successfully if I use this command, videos are downloaded

yt-dlp --referer "https:www.discoveryplus.com/" --add-header "Cookie:gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA@AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect" --batch-file links_da_scaricare.txt

ok, I try to download using Cygwin and I use this command

bash -x script.sh

But I get this error

$ bash -x script.sh
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas could not be read
+ IFS=
+ read -r url

script.sh have this code

#!/bin/bash

while IFS= read -r url; do
folder_name=$(echo "$url" | sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1')

    echo "folder_name: $folder_name"
    mkdir -p "/cygdrive/c/cygwin64/home/Administrator/$folder_name"
    echo "The script is running and creating folder: $folder_name" > ~/script.log
        yt-dlp --referer "https://www.discoveryplus.com/" --cookies "/cygdrive/c/cygwin64/home/Administrator/cookies.txt" -a "$url" -o "/cygdrive/c/cygwin64/home/Administrator/$folder_name/%(title)s.%(ext)s"
    done < /cygdrive/c/cygwin64/home/Administrator/links_da_scaricare.txt

links_da_scaricare.txt have these 3 lines - I use notepad to copy paste these links

https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

cookies.txt is this - is the same that I use in powershell command so it should download because with powershell I can do it but with bash i have problems, I don't know why

gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA@AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect

In other words script.sh should be do this

--> read urls from links_da_scaricare.txt and extract only these strings

citta-mortale
il-boss-delle-cerimonie

--> make folders with that names

--> download

https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas

in this folder

citta-mortale

--> download

https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

--> in this folder

il-boss-delle-cerimonie

Script creates the first folder but doesn't download anything inside it

Answer 1

You are looking to download files into a directory named according to the 6th field delimited by "/" in the url specification (between slash #5 and #6).

That is accomplished as

folder_name=$( echo "${url}" | cut -f6 -d/ )

Regarding sed, you must specify a delimiter character which does not conflict with any possible characters that might be found in the text string that you are trying to edit.

For URLs that always have "/", you cannot use "/" as that delimiter. You must choose something else. I tend to use either the "+" or the "|" depending on circumstances.

So to strip the prefix from the URL string, you would use

echo "${url}" | sed 's+https://www.discoveryplus.com/it/video/++'

to obtain

citta-mortale/stagione-1-episodio-1-strada-per-linferno
citta-mortale/stagione-1-episodio-2-non-in-kansas
il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello

You would make a second, similar, usage of sed to strip the tail end after the "/".

OR ... you can again use cut in this manner

id_string=$( echo "${url}" | cut -f7 -d/ )

to obtain

stagione-1-episodio-1-strada-per-linferno
stagione-1-episodio-2-non-in-kansas
stagione-5-il-quarantesimo-compleanno-di-nello