Python: replacing shell pipeline with an ssh connection

Question

I have the following nice bash command :

cat SomePythonScript.py | ssh remote_machine 'cat - | python'

that works very fine and that I want to write in Python. I tried with 'subprocess' but did not get that far. Can someone help me ?

from subprocess import  PIPE , Popen

p1 = Popen(["cat ", "SomePythonScript.py"], stdout=PIPE)
p2 = Popen(["remote_machine"], stdin=p1.stdout, stdout=PIPE)
p3 = Popen(["cat -", "python"], stdin=p2.stdout, stdout=PIPE)

p1.stdout.close()  
p2.stdout.close()  

output = p3.communicate()[0]

I tried also with 2 processes/ pipes

from  subprocess import PIPE , Popen  
p1 = Popen([["cat", "SomePythonScript.py"], stdout=PIPE)
p2 = Popen(["remote_machine","cat", "- python"], stdin=p1.stdout,    stdout=PIPE)
p1.stdout.close() 
output = p2.communicate()[0]

I would be very glad to any help, suggestion , advices, explanation solution... Thk in advance

Answer 1

There's nothing wrong with your use of Popen. The only things that's wrong is that you are missing the ssh command, and that you are trying to run three processes when there are only two in the bash command you're trying to mimic--'cat - | python' is just an argument to the ssh command.

The following should better mimic your bash command:

from subprocess import  PIPE , Popen

p1 = Popen(["cat", "SomePythonScript.py"], stdout=PIPE)
p2 = Popen(["ssh", "remote_machine", "cat - | python"], stdin=p1.stdout, stdout=PIPE)

p1.stdout.close()  

output = p2.communicate()[0]

Answer 2

Firstly, it's not worth replicating the useless use of cat even if you want the rest of your pipeline.

Secondly, it's conspicuous that although you included the entirely pointless cat command, you omitted the only command that actually does something - neither attempt runs ssh at all.

---

EDIT For future reference - since the accepted answer is still indulging in the same useless uses of cat, this is what the code ought to look like:

from subprocess import  PIPE , Popen

p = Popen(["ssh", "remote_machine", "python"], 
          stdin=open("SomePythonScript.py",'r'), stdout=PIPE)
output = p.communicate()[0]

And this is what the original shell code should have been in the first place:

< SomePythonScript.py ssh remote_machine python

Your original code contained

• p1 = Popen([["cat", "SomePythonScript.py"], stdout=PIPE)
  

  which has the same effect as open("SomePythonScript.py",'r'), but at the cost of running a child process for no benefit at all.

• p2 = Popen(["ssh", "remote_machine", "cat - | python"], ...
  

  which runs an additional redundant process on the remote machine. Since cat - is just copying its stdin to stdout, you can omit it completely