Why is Perl's chomp affecting the output of my print?

Question

It's been a couple months since I've been Perling, but I'm totally stuck on why this is happening...

I'm on OSX, if it matters.

I'm trying to transform lines in a file like

08/03/2011 01:00 PDT,1.11

into stdout lines like

XXX, 20120803, 0100, KWH, 0.2809, A, YYY

Since I'm reading a file, I want to chomp after each line is read in. However, when I chomp, I find my printing gets all messed up. When I don't chomp the printing is fine (except for the extra newline...). What's going on here?

while(<SOURCE>) {
    chomp;
    my @tokens = split(' |,');     # @tokens now [08/03/2011, 01:00, PDT, 1.11]

    my $converted_date = convertDate($tokens[0]);
    my $converted_time = convertTime($tokens[1]);

print<<EOF;
$XXX, $converted_date, $converted_time, KWH, $tokens[3], A, YYY
EOF
}

With the chomp call in there, the output is all mixed up:

, A, YYY10803, 0100, KWH, 1.11

Without the chomp call in there, it's at least printing in the right order, but with the extra new line:

XXX, 20110803, 0100, KWH, 1.11
, A, YYY

Notice that with the chomp in there, it's like it overwrites the newline "on top of" the first line. I've added the $|=1; autoflush, but don't know what else to do here.

Thoughts? And thanks in advance....

Answer 1

The lines of your input ends with CR LF. You're removing the LF only. A simple solution is to use the following instead of chomp:

s/\s+\z//;

You could also use the dos2unix command line tool to convert the files before passing them to Perl.

Answer 2

The problem is that you have DOS line-endings and are running on a Unix build of Perl.

One solution to this is to use PerlIO::eol. You may have to install it but there is no need for a use line in the program.

Then you can write

binmode ':raw:eol(LF)', $filehandle;

after which, regardless of the format or source of the file, the lines read will be terminated with the standard "\n".