Python subprocess.check_call() not recognising pushd and popd

Question

I'm on Ubuntu 15.04 (not by choice obviously) and Python 3.4.3 and I'm trying to execute something like the following.

subprocess.check_call("pushd /tmp", shell=True)

I need the shell=True because the actual code I'm trying to execute contains wildcards that need to be interpreted. However, this gives me the following error.

/usr/lib/python3.4/subprocess.py in check_call(*popenargs, **kwargs)
    559         if cmd is None:
    560             cmd = popenargs[0]
--> 561         raise CalledProcessError(retcode, cmd)
    562     return 0
    563 

CalledProcessError: Command 'pushd /tmp' returned non-zero exit status 127

I've tried doing the same thing on my Mac (El Capitan and Python 3.5.1) and it works perfectly. I've also tried executing subprocess.check_call("ls", shell=True) on the Ubuntu 15.04 with Python 3.4.3 (for sanity check), and it works fine. As a final sanity check, I've tried the command pushd /tmp && popd in Bash on the Ubuntu 15.04 and that works fine too. So somehow, on (my) Ubuntu 15.04 with Python 3.4.3, subprocess.check_call() does not recognise pushd and popd! Why?

Answer 1

You have two problems with your code. The first one is that the shell used by default is /bin/sh which doesn't support pushd and popd. In your question you failed to provide the whole error output, and at the top of it you should see the line:

/bin/sh: 1: popd: not found

The next time rememeber to post the whole error message, and not just the portion that you (incorrectly) think is relevant.

You can fix this by telling the subprocess module which shell to use via the executable argument:

>>> subprocess.check_call('pushd ~', shell=True, executable='/bin/bash')
~ ~
0

The second problem is that even with this you will get an error if you use multiple check_call calls:

>>> subprocess.check_call('pushd ~', shell=True, executable='/bin/bash')
~ ~
0
>>> subprocess.check_call('popd', shell=True, executable='/bin/bash')
/bin/bash: riga 0: popd: stack delle directory vuoto
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'popd' returned non-zero exit status 1

This is because every call to check_call starts a new subshells, and thus it doesn't matter whether you previously called pushd because the directory stack will always be empty.

Note that if you try to combine pushd and popd in a single call they do work:

>>> subprocess.check_call('pushd ~ && popd', shell=True, executable='/bin/bash')
~ ~
~
0

Now fact is, if you were thinking of using pushd and popd in that way from python... they are useless. That's because you can specify the current working directory via the cwd argument, and so you can keep track of the stack of working directories from python without having to rely on pushd and popd:

current_working_dirs = []

def pushd(dir):
    current_working_dirs.append(os.path.realpath(os.path.expanduser(dir)))

def popd():
    current_working_dirs.pop()


def run_command(cmdline, **kwargs):
    return subprocess.check_call(cmdline, cwd=current_working_dirs[-1], **kwargs)

Replace check_call('pushd xxx') with pushd('xxx') and check_call('popd') with popd and use run_command(...) instead of check_call(...).

---

As you suggest a more elegant solution would be to use a context manager:

class Pwd:
    dir_stack = []

    def __init__(self, dirname):
        self.dirname = os.path.realpath(os.path.expanduser(self.dirname))

    def __enter__(self):
        Pwd.dir_stack.append(self.dirname)
        return self

    def __exit__(self,  type, value, traceback):
        Pwd.dir_stack.pop()

    def run(self, cmdline, **kwargs):
        return subprocess.check_call(cmdline, cwd=Pwd.dir_stack[-1], **kwargs)

used as:

with Pwd('~') as shell:
    shell.run(command)
    with Pwd('/other/directory') as shell:
        shell.run(command2)   # runs in '/other/directory'
    shell.run(command3)       # runs in '~'