Runing into a deadlock when using Popen.wait()

Question

We need to execute a system command using subprocess.Popen().

As the output has a large data, when we tried to use Popen.wait() it got stuck.

pipe= subprocess.Popen(cmd, shell=False,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipe.wait()

So, we tried to put Popen.stdout in a variable pstdout. After which pstdout got closed and we are unable to fetch the content in that file.

pipe.stdout.closed
pstdout.closed

Can anyone help how to avoid that hung or how to reopen pstdout ?

Answer 1

Quoting python documentation popen on function Popen.wait():

Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use communicate() to avoid that.

and maybe you are run into deadlock. use communicate instead.

p= subprocess.Popen(cmd,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.communicate()