Rsync within expect script: error "No such file or directory"

Question

I want to execute rsync to update several directories on another machine, without typing the password for each one of them. I decided to try the expect extension by following some simple StackOverflow examples. See below a very simplified version of my script. The rsync command works correctly when executed by itself. However, when I run the script, I get the following output:

Remote password: 
spawn rsync -rv /home/john/TestDir/* [email protected]:/home/jim/TestDir
([email protected]) Password:
building file list ... 
rsync: [sender] link_stat "/home/john/TestDir/*" failed: No such file or directory (2)
done

sent 17 bytes  received 20 bytes  74.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]

#!/usr/bin/expect -d
 
read -s -p "Remote password: " PW
echo ""
expect <<EOF
    spawn rsync -rv  /home/john/TestDir/* [email protected]:/home/jim/TestDir
    expect "assword:"
    send "$PW\r"
    expect eof
EOF

Answer 1

The problem is that when you enter /path/* in the shell, it is the shell that expands this pattern to a list of filenames before passing it as an argument. However, the spawn command of expect does not do pattern expansion, so rsync receives a literal * and a file by that name does not exist.

There are several ways to work around it. One is to change the command to not use pattern expansion. @pynexj suggested removing the *, which works, but note that there is a difference between rsync -r foo/* bar and rsync -r foo/ bar: the latter will also copy hidden files (starting with .) while the former will not (because they are not matched by *).

However, you said your goal was to copy several directories without having to enter the password for each one. It's likely that a better solution to achieve this is to setup public key authentication, so you don't have to enter any passwords at all.

Here is one tutorial on how to configure this: https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server

Answer 2

For Expect ^{(which uses tcl)} the * char as in /home/john/TestDir/* has no special meaning. You can explicitly use a shell to run your command:

spawn bash -c "rsync -rv  /home/john/TestDir/* [email protected]:/home/jim/TestDir"
#     ^^^^^^^