Run a bash command in parallel with xargs

Question

I hope someone can help me with this simple problem. I want to run this command in parallel

windmill chrome test=./test http://www.google.ch

I was playing around with xargs and looked at the examples in the internet. However, I was not able to construct a xargs command to execute my mentioned command in parallel.

I tried the following

echo "chrome test=./test http://www.google.ch" | xargs -n 1 -P 2 windmill

which did not execute the right command meaning that windmill must have executed the wrong command because the output is not correct (default output of windmill for specifying wrong arguments).

Nevertheless, I got then another problem namely that the terminal/python complained about "socket.error: [Errno 48] Address already in use".

So when I run the windmill command in parallel by just openen for example two terminals and run in each terminal the windmill command and it works.

If xargs is not solution then I appreciate it a lot if you can pinpoint me to the correct way how to do it :)

Answer 1

xargs would not run your commands is parallel.

Why don't you just execute this in any bourne-compatible shell?

for i in 1 2; do windmill chrome test=./test http://www.google.ch & done

Answer 2

I've found that sometimes a program will have a problem binding to a recently released socket - the kernel tries to prevent this for security reasons. If you need to be able to rebind immediately, you probably should try SO_REUSEADDR or just bind to a different port each time.

Answer 3

You mention you want to run a command in parallel. That can only be done if the program itself is parallelized internally.

What you can do, however, is run multiple commands in parallel. Say you want to run these in parallel:

windmill chrome test=./test http://www.google.ch
windmill chrome test=./test http://www.google.se
windmill chrome test=./test http://www.google.no
windmill chrome test=./test http://www.google.de

Using GNU Parallel you can do:

parallel windmill chrome test=./test http://www.google.{} ::: ch se no de

Watch the intro videos to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 and walk through the tutorial (man parallel_tutorial). You command line with love you for it.

If that is not what you want, please re-phrase your question.