Thor: run command without capturing stdout or stderr, and fail on error

Question

I'm writing a Thor script to run some tests from a different tool i.e. running a shell command. I'd like the stdout and stderr from the command to continuously stream out into my console.

First attempt was to just use backticks, but naturally the stdout/stderr are not printed (rather, stdout is captured in the return value).

desc "mytask", "my description"
def mytask
  `run-my-tests.sh`
end

My next approach was to use Open3 as in:

require "open3"
desc "mytask", "my description"
def mytask
  Open3.popen3("run-my-tests.sh") do |stdin, stdout, stderr|
    STDOUT.puts(stdout.read())
    STDERR.puts(stderr.read())
  end
end

However, the above approach will get the whole output from both stdout and stderr and only print at the end. Un my use case, I'd rather see the output of failing and passing tests as it becomes available.

From http://blog.bigbinary.com/2012/10/18/backtick-system-exec-in-ruby.html, I saw that we can read the streams by chunks i.e. with gets() instead of read(). For example:

require "open3"
desc "mytask", "my description"
def mytask
  Open3.popen3(command) do |stdin, stdout, stderr|
    while (out = stdout.gets()) || err = (stderr.gets())
      STDOUT.print(out) if out
      STDERR.print(err) if err
    end
    exit_code = wait_thr.value
    unless exit_code.success?
      raise "Failure"
    end
  end
end

Does it look like the best and cleanest approach? Is it an issue that I have to manually try to print stdout before stderr?

Answer 1

I'm using IO.popen for similar task, like so: IO.popen([env, *command]) do |io| io.each { |line| puts ">>> #{line}" } end To capture stderr I'd just redirect it to stdout command = %w(run-my-tests.sh 2>&1)

Update I've constructed a script using Open3::popen3 to capture stdout and stderr separately. It obviously has a lot of room form improvement, but basic idea hopefully is clear.

require 'open3'

command = 'for i in {1..5}; do echo $i; echo "$i"err >&2; sleep 0.5; done'
stdin, stdout, stderr, _command_thread = Open3.popen3(command)

reading_thread = Thread.new do
  kilobyte = 1024

  loop do
    begin
      stdout.read_nonblock(kilobyte).lines { |line| puts "stdout >>> #{line}" }
      stderr.read_nonblock(kilobyte).lines { |line| puts "stderr >>> #{line}" }
    rescue IO::EAGAINWaitReadable
      next
    rescue EOFError
      break
    end

    sleep 1
  end
end

reading_thread.join
stdin.close
stdout.close
stderr.close

Answer 2

Seems to me like the simplest way to run a shell command and not try to capture the stdout or stderr (instead, let them bubble up as they come) was something like:

def run *args, **options
  pid = spawn(*args, options)
  pid, status = Process.wait2(pid)
  exit(status.exitstatus) unless status.success?
end

The problem with backticks or system() is that the former captures the stdout and the latter only returns whether the command succeeded or not. spawn() is a more informative alternative to system(). I'd rather have my Thor script tool fail as if it was merely a wrapper for those shell commands.

Answer 3

In case you end up here because you just want to fail on error, you can add the abort_on_failure: true option to the run command, as such:

run "bundle install", abort_on_failure: true