plumbum.commands.processes.ProcessExecutionError: for commands which return null

Question

The shell command I want to run, which returns nothing:

echo helloWorld | grep 'dummy'

---

plumbum version:

Following line works:

out=(echo["helloWorld"] | grep["h"])().strip()

---

But following line does not, what might be the reason?

out=(echo["helloWorld"] | grep["dummy"])().strip()
print(out)

Error I am having:

Traceback (most recent call last):
  File "dd.py", line 6, in <module>
    out=(echo["helloWorld"] | grep["dummy"])().strip()
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 103, in __call__
    return self.run(args, **kwargs)[1]
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 240, in run
    return p.run()
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 201, in runner
    return run_proc(p, retcode, timeout)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/processes.py", line 232, in run_proc
    return _check_process(proc, retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/processes.py", line 23, in _check_process
    proc.verify(retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/commands/base.py", line 412, in verify
    dstproc_verify(retcode, timeout, stdout, stderr)
  File "/home/user/venv/lib/python3.5/site-packages/plumbum/machines/base.py", line 26, in verify
    stderr)
plumbum.commands.processes.ProcessExecutionError: Command line: ['/bin/grep', 'dummy']
Exit code: 1

[Q] How could I fix this error?

Answer 1

This happens because the exit status of grep is 1 if it does not find anything, as described in its manual

You can try it in command line if you wish:

echo helloWorld | grep h; echo $?
echo helloWorld | grep x; echo $?

Will result in

helloWorld
0
1

Ways to circumvent this are described in another nice answer, e.g.

echo helloWorld | grep x | cat

will yield 0 as status. But unfortunately plumbum does a local pipe mechanism so grep output goes to plumbum, and then plumbum pipes it to the next command - meaning the cat can't swallow the exit code 1, there will be an exception thrown before it.

So my two ideas are to either create a shell script to run a grep never returning error on search without results:

#!/bin/bash
grep "$@" || test $? = 1

and execute this instead of the grep (called c1grep in the original answer), or to add a try/except block around your pipe code and manually handle the exit code 1 (ProcessExecutionError).

Answer 2

The other answer is correct but less than optimal - The problem can be solved in plumbum, without the need for an external script that wraps grep, and without having to catch the exception.

The idea is to use plumbum's run method to to do two things:

• Force plumbum to accept any exit code from the called program (by default, an exception is raised if the returned error code is not zero).
• Capture a tuple containing the error code, the standard output, and the standard error of the called program, and not just the standard output as plumbum normally does.

Working example:

for s in ["hello","dummy"]:
    exitcode,stdout,stderr = (echo["helloWorld"] | grep[s]).run (retcode=None)

    if exitcode == 0:
        print (stdout.strip())

    if exitcode != 0:
        print (f"string {s} not present")
        print (stderr)

The previous code returns:

helloWorld
string dummy not present

Without raising any exception.