`last` while reading file from external command crashes perl

Question

I have the following code:

#!/usr/bin/env perl

use 5.0360;
use warnings FATAL => 'all';
use autodie ':default';
use Devel::Confess 'color'; # not essential, but better error reporting

open my $view, "zcat a.big.file.vcf.gz|"; # zcat or bcftools
while (<$view>) {
    next unless /^#CHROM\t/;
    last;
}
close $view;

the above code crashes with the error

Can't close(GLOB(0x55adfa96ebf8)) filehandle: '' at (eval 11)[/home/con/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/Fatal.pm:1683] line 74
 at (eval 11)[/home/con/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/Fatal.pm:1683] line 74.
    main::__ANON__[(eval 11)[/home/con/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/Fatal.pm:1683]:86](GLOB(0x55adfa96ebf8)) called at mwe.pl line 13
Command exited with non-zero status 255

However, if I comment out last the code runs without a problem, however, the file is huge and this makes a significant difference in running time.

The code also works if I remove close $view but close is proper practice.

How can I run the code with both last and close $view?

Answer 1

When you last out of reading that process (zcat here) and close the pipe, before the process is done writing, the process gets a SIGPIPE

Closing the read end of a pipe before the process writing to it at the other end is done writing results in the writer receiving a SIGPIPE

So as close then waits on it it gets a non-zero, and returns false (as seen below). That's all. The rest -- the program "crashing" -- is up to autodie, which throws an exception. Without autodie (or fatal warnings)

use warnings;
use strict;
use feature 'say';

use Scalar::Util qw(openhandle);

my $file = shift // die "Usage: $0 file\n";

open my $view, "zcat $file |" or die "Can't pipe-open zcat: $!";

while (<$view>) {
    next unless /^#CHROM\t/;  # (added to my test file)
    say "Matched --> $_";
    last;
}

say "Filehandle good? --> ", openhandle($view) // 'undef';  # GLOB(...)
    
close $view or warn $!  
    ? "Error closing pipe: $!" 
    : "Command exit status: $?";                            # 13

say "Program received a signal ", $? & 127  if $? & 127;    # 13

I get Command exit status: 13, so close didn't return true while $! is false. This indicates that the only issue was the non-zero status

If the filehandle came from a piped open, close returns false if one of the other syscalls involved fails or if its program exits with non-zero status. If the only problem was that the program exited non-zero, $! will be set to 0.

We did get a signal, 13 (for sigpipe, see man 7 signal), what terminated the program and so there was no particular exit code ($? >> 8 is indeed zero), and no core was dumped ($? & 128 is zero). See $? in perlvar

Since the exit status is non-zero close returns false and autodie throws its exception.

---

So what to do with this?

That close must stay and be checked, of course.

Even if the SIGPIPE sent to zcat could be ignored, as I've seen claimed in some docs, you wouldn't want that -- it's there on purpose, to let the writer know that there are no readers so that it can stop!

Finally, it is autodie that kills the program, and it can be disabled lexically. (This satisfies the "need" for autodie stated in a comment.) So put this pipe reading and early close in a block

READ_FROM_PIPE: { 
    no autodie qw(close);
    # do the pipe-open, read from the pipe, close() it...
};
# rest of code 

Don't forget to suitably adjust your other error handling in this code.

(I've had weird experience with no autodie "leakage" out of its scope, see here. But that was a different case, and fixed by autodie 2.30 so hopefully not of concern.)

Another option is to wrap all this, or just the close(), in eval instead. This is considered good practice, they say in docs. Then see how to work with autodie exceptions.